“Chao Xing 2016-02-01”版本间的差异
来自cslt Wiki
（以“LAST WEEK ---- Done: *Multi-task transfer learning during transfer-step pre-train. *Find some strategies among each hyper-parameters. **Steps determine the train pa...”为内容创建页面） |
|||
| 第2行： | 第2行： | ||
---- | ---- | ||
Done: | Done: | ||
| − | *Multi-task transfer learning during transfer | + | *Multi-task transfer learning during transfer step pretrain. |
*Find some strategies among each hyper-parameters. | *Find some strategies among each hyper-parameters. | ||
**Steps determine the train parameters information content, which means larger steps need larger iterations and larger learning rate. | **Steps determine the train parameters information content, which means larger steps need larger iterations and larger learning rate. | ||
2016年2月1日 (一) 02:21的最后版本
LAST WEEK
Done:
- Multi-task transfer learning during transfer step pretrain.
- Find some strategies among each hyper-parameters.
- Steps determine the train parameters information content, which means larger steps need larger iterations and larger learning rate.
- Learning rate should not decay in this program.
- Larger learning may obtain better performance.
THIS WEEK
Want to do:
- In my test:
- Steps must consist among source images pretrain, target images pretrain, transfer pretrain and fine tune.
- In transfer pretrain learning rate could have a larger number lead iteration count could be small, for last two days experiment, I choose 1e-2 as transfer learning rate.
- In fine tune step, the learning rate should be as small as possible but not too small, in this experiment I choose 1e-3.
- So this week I will focus on this experiment.
