14-10-19 Dongxu Zhang

Accomplished this week

• Train LSTM-Rnn LM with 200MB corpus(vocabulary 10k, classes 100). when using 2 kernels, it takes aroung 200min per epoch.
• Train 5-gram LM using Baiduzhidao_corpus(~30GB after preprocess) with new lexicon. There is a mistake when counted possiblity after merge.
• An idea occured to me which may improve word2vec with much more semantic information. But there is huge computation complexity problem that bothers me, which I wish we can discuss.
• Read paper "Learning Long-Term Dependencies with Gradient Descent is Difficult". Still in progress.

Next week

• 1. Test LSTM-Rnn LM.
• 2. Finished building lexion.
• 3. Understand the paper.
• 4. May have time to achieve my baseline idea on text8.