Zhiyuan Tang 15-07-07

Last week:

Why dropout didn't help Rectifier in my experiment?

Leaky Rectifier has been realized, while it didn't seem to be better;

Temporal Rectifier, where each neuron has its own threshhold which changes along time in order to remeber the temporal relationship between contexts, has almost been done. Even it has no novelty, I have learned much about Kaldi.

This week:

accomplish the draft whose temporary title is "Deep Neural Network Acoustic Model with Language Vector";

at least do some conclusion on RECTIFIER, then move on.