Chao Xing 2016-02-01

LAST WEEK

---

Done:

• Multi-task transfer learning during transfer-step pre-train.
• Find some strategies among each hyper-parameters.
  • Steps determine the train parameters information content, which means larger steps need larger iterations and larger learning rate.
  • Learning rate should not decay in this program.
  • Larger learning may obtain better performance.

THIS WEEK

---

Want to do:

• In my test:
  • Steps must consist among source images pretrain, target images pretrain, transfer pretrain and fine tune.
  • In transfer pretrain learning rate could have a larger number lead iteration count could be small, for last two days experiment, I choose 1e-2 as transfer learning rate.
  • In fine tune step, the learning rate should be as small as possible but not too small, in this experiment I choose 1e-3.
• So this week I will focus on this experiment.