cslt Wiki - 用户贡献 [zh-cn]

http://index.cslt.org/mediawiki/api.php?action=feedcontributions&feedformat=atom&user=Renshipan cslt Wiki - 用户贡献 [zh-cn] 2026-04-17T09:24:07Z 用户贡献 MediaWiki 1.23.3 http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-04T07:41:23Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-04T07:40:47Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0) '&quot'<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-04T07:40:15Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0) ‘&quot’<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-04T07:37:24Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-04T07:36:25Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code" were changed to ‘&quot;’<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-04T07:35:50Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code" were changed to ‘&quot’<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T02:04:31Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> |-<br /> |Shipan Ren <br /> || <br /> * checked experimental results and found some small problems,like this [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/9/99/Replacement.docx]<br /> * wrote the first draft paper of ViVi_NMT [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/8/80/ViVi_NMT.docx]<br /> * pushed the baseline code,readme file and user manual to cslt github(https://github.com/CSLT-THU/CSLT_NMT)<br /> || <br /> * read some translating phrases papers<br /> * learn more about the memory mechanism <br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * Clean up the code of predict.py<br /> <br /> || <br /> * configuration environment of TF-0.12 or update the code to adapt TF-1.0<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/%E6%96%87%E4%BB%B6:Replacement.docx 文件:Replacement.docx 2017-09-04T02:03:42Z

<p>Renshipan：</p> <hr /> <div></div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T02:00:57Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> |-<br /> |Shipan Ren <br /> || <br /> * checked experimental results and found some small problems.<br /> * wrote the first draft paper of ViVi_NMT [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/8/80/ViVi_NMT.docx]<br /> * pushed the baseline code,readme file and user manual to cslt github(https://github.com/CSLT-THU/CSLT_NMT)<br /> || <br /> * read some translating phrases papers<br /> * learn more about the memory mechanism <br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * Clean up the code of predict.py<br /> <br /> || <br /> * configuration environment of TF-0.12 or update the code to adapt TF-1.0<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T02:00:06Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> |-<br /> |Shipan Ren <br /> || <br /> * checked experimental results and found some small problems，like follows：<br /> sentence before tokenize：<br /> "If no more people smoked, this rate would increase to at least 50%," says André Beaulieu, <br /> spokesman for the Canadian Cancer Society.<br /> sentence after tokenize：<br /> &quot; if no more people smoked , this rate would increase to at least 50 % , &quot; says andré beaulieu , <br /> spokesman for the canadian cancer society .<br /> It was just a equivalent replacement<br /> * wrote the first draft paper of ViVi_NMT [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/8/80/ViVi_NMT.docx]<br /> * pushed the baseline code,readme file and user manual to cslt github(https://github.com/CSLT-THU/CSLT_NMT)<br /> || <br /> * read some translating phrases papers<br /> * learn more about the memory mechanism <br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * Clean up the code of predict.py<br /> <br /> || <br /> * configuration environment of TF-0.12 or update the code to adapt TF-1.0<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/%E6%96%87%E4%BB%B6:ViVi_NMT.docx 文件:ViVi NMT.docx 2017-09-04T01:58:56Z

<p>Renshipan：</p> <hr /> <div></div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T01:58:38Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> |-<br /> |Shipan Ren <br /> || <br /> * checked experimental results and found some small problems，like follows：<br /> sentence before tokenize：<br /> "If no more people smoked, this rate would increase to at least 50%," says André Beaulieu, <br /> spokesman for the Canadian Cancer Society.<br /> sentence after tokenize：<br /> &quot; if no more people smoked , this rate would increase to at least 50 % , &quot; says andré beaulieu , <br /> spokesman for the canadian cancer society .<br /> It was just a equivalent replacement<br /> * wrote the first draft paper of ViVi_NMT <br /> * pushed the baseline code,readme file and user manual to cslt github(https://github.com/CSLT-THU/CSLT_NMT)<br /> || <br /> * read some translating phrases papers<br /> * learn more about the memory mechanism <br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * Clean up the code of predict.py<br /> <br /> || <br /> * configuration environment of TF-0.12 or update the code to adapt TF-1.0<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T01:53:02Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> |-<br /> |Shipan Ren <br /> || <br /> * checked experimental results and found some small problems，like follows：<br /> sentence before tokenize：<br /> "If no more people smoked, this rate would increase to at least 50%," says André Beaulieu, <br /> spokesman for the Canadian Cancer Society.<br /> sentence after tokenize：<br /> &quot; if no more people smoked , this rate would increase to at least 50 % , &quot; says andré beaulieu , <br /> spokesman for the canadian cancer society .<br /> It is just a equivalent replacement<br /> * wrote the first draft paper of ViVi_NMT <br /> * pushed the baseline code,readme file and user manual to cslt github(https://github.com/CSLT-THU/CSLT_NMT)<br /> || <br /> * read some translating phrases papers<br /> * learn more about the memory mechanism <br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * Clean up the code of predict.py<br /> <br /> || <br /> * configuration environment of TF-0.12 or update the code to adapt TF-1.0<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T01:52:07Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> |-<br /> |Shipan Ren <br /> || <br /> * checked experimental results and found some few small problems，like follows：<br /> sentence before tokenize：<br /> "If no more people smoked, this rate would increase to at least 50%," says André Beaulieu, <br /> spokesman for the Canadian Cancer Society.<br /> sentence after tokenize：<br /> &quot; if no more people smoked , this rate would increase to at least 50 % , &quot; says andré beaulieu , <br /> spokesman for the canadian cancer society .<br /> It is just a equivalent replacement<br /> * wrote the first draft paper of ViVi_NMT <br /> * pushed the baseline code,readme file and user manual to cslt github(https://github.com/CSLT-THU/CSLT_NMT)<br /> || <br /> * read some translating phrases papers<br /> * learn more about the memory mechanism <br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * Clean up the code of predict.py<br /> <br /> || <br /> * configuration environment of TF-0.12 or update the code to adapt TF-1.0<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-9-4 NLP Status Report 2017-9-4 2017-09-04T01:32:47Z

<p>Renshipan：以“{| class="wikitable" !Date !! People !! Last Week !! This Week |- | rowspan="6"|2017/8/14 |Jiyuan Zhang || *code refactoring *wrote a document[http://cslt.riit.tsing...”为内容创建页面</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren <br /> || <br /> * read the released information of other toolkits for nmt<br /> * cleaned up the code<br /> * wrote the documents [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/5/5c/Manual_V1.0.docx] [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/3/38/Manual_V0.10.docx]<br /> || <br /> * write the papers of our baseline system<br /> * read augmented nmt code<br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * learn the source code of the mode<br /> <br /> || <br /> * learn the source code of seq2seq model and learn tensorflow<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-09-01T02:04:18Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Jiayu Guo || 10:00 || 20:00 || 10 || <br /> * clean up the code<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-31T13:21:29Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/31<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-31T13:18:46Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jiayu Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jiayu Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jiayu Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jiayu Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * check experimental results<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jiayu Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Jiayu Guo || 8:10 || 21:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/29<br /> |Jiayu Guo || 11:00 || 21:00 || 10 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> | rowspan="1"|2017/08/30<br /> |Jiayu Guo || 11:30 || 21:00 || 9 || <br /> * learn VV model<br /> |-<br /> | rowspan="1"|2017/08/28<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the paper of ViVi_NMT(version 1.0)<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-28 NLP Status Report 2017-8-28 2017-08-28T07:50:59Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document[http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:VvPoem.docx]<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren <br /> || <br /> * read the released information of other toolkits for nmt<br /> * cleaned up the code<br /> * wrote the documents [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/5/5c/Manual_V1.0.docx] [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/3/38/Manual_V0.10.docx]<br /> || <br /> * write the papers of our baseline system<br /> * read augmented nmt code<br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * learn the source code of the mode<br /> <br /> || <br /> * learn the source code of seq2seq model and learn tensorflow<br /> |-<br /> |-<br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/%E6%96%87%E4%BB%B6:Manual_V0.10.docx 文件:Manual V0.10.docx 2017-08-28T07:50:32Z

<p>Renshipan：user manual of nmt baseline 0.10</p> <hr /> <div>user manual of nmt baseline 0.10</div>

Renshipan http://index.cslt.org/mediawiki/index.php/%E6%96%87%E4%BB%B6:Manual_V1.0.docx 文件:Manual V1.0.docx 2017-08-28T07:46:48Z

<p>Renshipan：user manual of nmt baseline system</p> <hr /> <div>user manual of nmt baseline system</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-21 NLP Status Report 2017-8-21 2017-08-28T07:44:30Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *done some work about code refactoring for poem system <br /> || <br /> *plan to complete code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * organized all the experimental results(our baseline system,Moses,THUMT) [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/8/89/Nmt_baseline.xlsx]<br /> * trained and tested translation models（Toolkit:THUMT ）<br /> * compared with our system<br /> ||<br /> * prepare to release the baseline system（tensorflow1.0 version）<br /> |-<br /> <br /> |Jiayu Guo||<br /> * process data and run model;<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(pairs of sentence/the quality——the modern language text includes context information).<br /> || <br /> *plan to read source code of seq2seq model and learn tensorflow;<br /> *plan to read a paper named Automatic Long Sentence Segmentation for NMT<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/%E6%96%87%E4%BB%B6:Nmt_baseline.xlsx 文件:Nmt baseline.xlsx 2017-08-28T07:43:36Z

<p>Renshipan：nmt baseline experimental results</p> <hr /> <div>nmt baseline experimental results</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-28 NLP Status Report 2017-8-28 2017-08-28T06:31:53Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren <br /> || <br /> * read the released information of other toolkits for nmt<br /> * cleaned up the code<br /> * wrote the documents <br /> || <br /> * write the papers of our baseline system<br /> * read augmented nmt code<br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * process data and run model;<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(pairs of sentence/the quality——the modern language text includes context information).<br /> || <br /> *plan to read source code of seq2seq model and learn tensorflow;<br /> *plan to read a paper named Automatic Long Sentence Segmentation for NMT<br /> |-<br /> |-<br /> |zhangshuai <br /> || <br /> * learn model source code<br /> || <br /> * learn tensorflow and source code<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-28T06:28:36Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jia Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jia Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jia Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jia Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jia Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> <br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-28T06:25:55Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 9:00|| 20:00 || 11 ||<br /> * test results.<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jia Guo || 9:30 || 21:30 || 12 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Jia Guo || 9:10 || 23:00 || 9.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> <br /> | rowspan="1"|2017/08/22<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * cleaned up the code<br /> |-<br /> | rowspan="1"|2017/08/22<br /> |Jia Guo || 9:00 || 22:00 || 12 || <br /> * read the source code <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * wrote the documents<br /> <br /> |-<br /> | rowspan="1"|2017/08/23<br /> |Jia Guo || 9:00 || 22:00 || 11 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * wrote the documents<br /> |-<br /> | rowspan="1"|2017/08/24<br /> |Jia Guo || 9:10 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> | rowspan="1"|2017/08/25<br /> |Jia Guo || 8:50 || 22:00 || 10.5 || <br /> * read the source code and learn tensorflow<br /> |-<br /> <br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-28 NLP Status Report 2017-8-28 2017-08-28T06:22:16Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *code refactoring<br /> *wrote a document<br /> || <br /> <br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren <br /> || <br /> * wrote the documents <br /> * cleaned up the code<br /> || <br /> * write the papers of our baseline system<br /> * read augmented nmt code<br /> |-<br /> <br /> <br /> |Jiayu Guo||<br /> * process data and run model;<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(pairs of sentence/the quality——the modern language text includes context information).<br /> || <br /> *plan to read source code of seq2seq model and learn tensorflow;<br /> *plan to read a paper named Automatic Long Sentence Segmentation for NMT<br /> |-<br /> |-<br /> |zhangshuai <br /> || <br /> * learn model source code<br /> || <br /> * learn tensorflow and source code<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T11:40:05Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T11:39:33Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/21<br /> |Shipan Ren || 10:00 || 22:00 || 12 || <br /> * read the released information of other translation systems<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T11:29:16Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> <br /> <br /> <br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-<br /> <br /> | rowspan="1"|2017/08/15<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/16<br /> |Jiayu Guo || 10:00|| 23:00 || 10||<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> |-<br /> <br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> <br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T11:22:00Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> <br /> |-<br /> <br /> | rowspan="1"|2017/08/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> <br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> <br /> |-<br /> | rowspan="1"|2017/08/15<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> <br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/17<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> <br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-21 NLP Status Report 2017-8-21 2017-08-21T05:16:41Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *done some work about code refactoring for poem system <br /> || <br /> *plan to complete code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * organized all the experimental results(our baseline system,Moses,THUMT) <br /> * trained and tested translation models（Toolkit:THUMT ）<br /> * compared with our system<br /> ||<br /> * prepare to release the baseline system（tensorflow1.0 version）<br /> |-<br /> <br /> |Jiayu Guo||<br /> * process data and run model;<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> || <br /> * read source code of seq2seq model;<br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-21 NLP Status Report 2017-8-21 2017-08-21T05:09:46Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *done work about code refactoring for poem system <br /> || <br /> *plan to complete code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * organized all the experimental results(our baseline system,Moses,THUMT) <br /> * train translation models by using THUMT <br /> * test the bleu of these models<br /> * compare with our system<br /> ||<br /> * prepare to release the baseline system（tensorflow1.0 version）<br /> |-<br /> <br /> |Jiayu Guo||<br /> * process data and run model;<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> || <br /> * read source code of seq2seq model;<br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T03:21:28Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-de <br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-de dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset:WMT2014 en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read papers about memory-augmented nmt<br /> <br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> <br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document <br /> |<br /> <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read memory-augmented nmt code <br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints of en-fr dataset<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * looked for the performance(the bleu value) of other seq2seq models<br /> * datasets:WMT2014 en-de and en-fr <br /> <br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * learn moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * installed and built Moses on the server <br /> <br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation model and test it<br /> * dataset:zh-en small<br /> * test if moses can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * toolkit: Moses<br /> <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train statistical machine translation models and test it <br /> * dataset:zh-en big,WMT2014 en-de,WMT2014 en-fr<br /> <br /> |-<br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * collate experimental results<br /> * compare our baseline model with Moses <br /> <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> <br /> |-<br /> | rowspan="1"|2017/08/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read paper about THUMT<br /> <br /> |-<br /> | rowspan="1"|2017/08/12<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> <br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * read THUMT manual and learn how to use it<br /> <br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en small<br /> * test if THUMT can work normally<br /> <br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * code automation scripts to process data,train model and test model <br /> * train translation models and test them<br /> * toolkit: THUMT<br /> * dataset:zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * test translation models by using single reference and multiple reference <br /> * organize all the experimental results(our baseline system,Moses,THUMT)<br /> <br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T02:24:07Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> * (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * datasets:WMT2014 en-de and en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> <br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document<br /> <br /> |<br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> |-<br /> <br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-s<br /> | rowspan="1"|2017/08/12<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T02:21:48Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * dataset：zh-en big<br /> <br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> * datasets:WMT2014 en-de and en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> <br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document<br /> <br /> |<br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> |-<br /> <br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-s<br /> | rowspan="1"|2017/08/12<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T02:19:48Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> dataset：zh-en small<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> (reason: the test set and the train set of small data set are similar in content and style) <br /> <br /> |-<br /> | rowspan="1"|2017/07/12<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> dataset：zh-en big<br /> <br /> <br /> |-<br /> | rowspan="1"|2017/07/13<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> * I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> <br /> |-<br /> | rowspan="1"|2017/07/14<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> <br /> |-<br /> | rowspan="1"|2017/07/17<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * downloaded the wmt2014 data sets and processed it<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> <br /> |-<br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> <br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * processed data<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> datasets:WMT2014 en-de and en-fr datasets<br /> |-<br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> <br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document<br /> <br /> |<br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> |-<br /> <br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-s<br /> | rowspan="1"|2017/08/12<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T02:02:12Z

<p>Renshipan：</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * trained translation models using tf1.0 baseline and tf0.1 baseline perspectively<br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * tested these checkpoints<br /> * found the new version takes less time <br /> * found these two versions have similar complexity and bleu values <br /> * found that the bleu is still good when the model is over fitting .<br /> (reason: the test set and the train set of small data set are similar in content and style) <br /> |-<br /> <br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> <br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> <br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document<br /> <br /> |<br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> |-<br /> <br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-s<br /> | rowspan="1"|2017/08/12<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/Schedule Schedule 2017-08-21T01:53:45Z

<p>Renshipan：/* Daily Report */</p> <hr /> <div>=NLP Schedule=<br /> <br /> ==Members==<br /> <br /> ===Current Members===<br /> <br /> * Yang Feng (冯洋)<br /> * Jiyuan Zhang （张记袁）<br /> * Aodong Li (李傲冬)<br /> * Andi Zhang (张安迪)<br /> * Shiyue Zhang (张诗悦)<br /> * Li Gu (古丽)<br /> * Peilun Xiao (肖培伦)<br /> * Shipan Ren (任师攀)<br /> * Jiayu Guo (郭佳雨)<br /> <br /> ===Former Members===<br /> * '''Chao Xing (邢超)''' : FreeNeb<br /> * '''Rong Liu (刘荣)''' : 优酷<br /> * '''Xiaoxi Wang (王晓曦)''' : 图灵机器人<br /> * '''Xi Ma (马习)''' : 清华大学研究生<br /> * '''Tianyi Luo (骆天一)''' ： phd candidate in University of California Santa Cruz<br /> * '''Qixin Wang (王琪鑫)''' : MA candidate in University of California<br /> * '''DongXu Zhang (张东旭)''': --<br /> * '''Yiqiao Pan (潘一桥)''' ： MA candidate in University of Sydney <br /> * '''Shiyao Li （李诗瑶）''' : BUPT<br /> * '''Aiting Liu (刘艾婷)''' : BUPT<br /> <br /> ==Work Progress==<br /> ===Daily Report===<br /> <br /> {|class="wikitable"<br /> ! Date !! Person !! start!! leave !! hours ||status<br /> |-<br /> | rowspan="2"|2017/04/02<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/03<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/04<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/05<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/06<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/07<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/08<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/09<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/10<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/11<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/12<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/13<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/14<br /> |Andy Zhang||9:30 ||18:30 ||8 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="2"|2017/04/15<br /> |Andy Zhang||9:00 ||15:00 ||6 || <br /> *preparing EMNLP<br /> |-<br /> |Peilun Xiao || || || ||<br /> |-<br /> | rowspan="1"|2017/04/18<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Pick up new task in news generation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/19<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/20<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/21<br /> |Aodong Li||12:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/24<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Adjust literature review focus<br /> |-<br /> | rowspan="1"|2017/04/25<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/26<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/04/27<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Try to reproduce sc-lstm work<br /> |-<br /> | rowspan="1"|2017/04/28<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Transfer to new task in machine translation and do literature review<br /> |-<br /> | rowspan="1"|2017/04/30<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/01<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review<br /> |-<br /> | rowspan="1"|2017/05/02<br /> |Aodong Li||11:00 ||20:00 ||8 || <br /> *Literature review and code review<br /> |-<br /> | rowspan="1"|2017/05/06<br /> |Aodong Li||14:20 ||17:20||3 || <br /> *Code review<br /> |-<br /> | rowspan="1"|2017/05/07<br /> |Aodong Li||13:30 ||22:00||8 || <br /> *Code review and experiment started, but version discrepancy encountered<br /> |-<br /> | rowspan="1"|2017/05/08<br /> |Aodong Li||11:30 ||21:00 ||8 || <br /> *Code review and version discrepancy solved<br /> |-<br /> | rowspan="1"|2017/05/09<br /> |Aodong Li||13:00 ||22:00 ||9 || <br /> *Code review and experiment<br /> *details about experiment: <br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 42.56<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> *Entry procedures<br /> *Machine Translation paper reading<br /> |-<br /> | rowspan="1"|2017/05/10<br /> |Aodong Li || 13:30 || 22:00 || 8 || <br /> *experiment setting: <br /> small data, <br /> 1st and 2nd translator uses the different training data, counting 22000 and 22017 seperately<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 36.67 (36.67 is the model at 4750 updates, but we use model at 3000 updates to<br /> prevent the case of overfitting, to generate the 2nd translator's training data, for <br /> which the BLEU is 34.96)<br /> best result of our model: 29.81<br /> This may suggest that that using either the same training data with 1st translator or different<br /> one won't influence 2nd translator's performance, instead, using the same one may<br /> be better, at least from results. But I have to give a consideration of a smaller size <br /> of training data compared to yesterday's model.<br /> *code 2nd translator with constant embedding<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *Configure environment <br /> *Run tf_translate code<br /> *Read Machine Translation paper<br /> |-<br /> | rowspan="1"|2017/05/11<br /> |Aodong Li || 13:00 || 21:00|| 8 || <br /> *experiment setting:<br /> small data, <br /> 1st and 2nd translator uses the same training data, <br /> 2nd translator uses '''constant untrainable embedding''' imported from 1st translator's decoder<br /> *results (BLEU):<br /> BASELINE: 43.87<br /> best result of our model: 43.48<br /> Experiments show that this kind of series or cascade model will definitely impair the final perfor-<br /> mance due to information loss as the information flows through the network from <br /> end to end. Decoder's smaller vocabulary size compared to encoder's demonstrate<br /> this (9000+ -> 6000+).<br /> The intention of this experiment is looking for a map to solve meaning shift using 2nd translator,<br /> but result of whether the map is learned or not is obscured by the smaller vocab size <br /> phenomenon.<br /> *literature review on hierarchical machine translation<br /> |-<br /> | rowspan="1"|2017/05/12<br /> |Aodong Li||13:00 ||21:00 ||8 || <br /> *Code double decoding model and read multilingual MT paper<br /> |-<br /> | rowspan="1"|2017/05/13<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> *read machine translation paper <br /> *learne lstm model and seq2seq model <br /> |-<br /> | rowspan="1"|2017/05/14<br /> |Aodong Li || 10:00 || 20:00 || 9 || <br /> *Code double decoding model and experiment<br /> *details about experiment: <br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: 43.53<br /> *NEXT: 2nd translator uses '''trained constant embedding'''<br /> |-<br /> | rowspan="1"|2017/05/15<br /> |Shipan Ren || 9:30 || 19:00 || 9.5 || <br /> * understand the difference between lstm model and gru model<br /> * read the implement code of seq2seq model<br /> |-<br /> | rowspan="2"|2017/05/17<br /> |Shipan Ren || 9:30 || 19:30 || 10 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 13:30 || 24:00 || 9|| <br /> * code and debug double-decoder model<br /> * alter 2017/05/14 model's size and will try after nips<br /> |-<br /> | rowspan="2"|2017/05/18<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * read neural machine translation paper<br /> * read tf_translate code<br /> |-<br /> |Aodong Li || 12:30 || 21:00 || 8 || <br /> * train double-decoder model on small data set but encounter decode bugs<br /> |-<br /> | rowspan="1"|2017/05/19<br /> |Aodong Li || 12:30 || 20:30 || 8 || <br /> * debug double-decoder model<br /> * the model performs well on develop set, but performs badly on test data. I want to figure out the reason.<br /> |-<br /> | rowspan="1"|2017/05/21<br /> |Aodong Li || 10:30 || 18:30 || 8 || <br /> *details about experiment: <br /> hidden_size = 700 (500 in prior)<br /> emb_size = 510 (310 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.21'''<br /> But only one checkpoint outperforms the baseline, the other results are commonly under 43.1<br /> * debug double-decoder model<br /> |-<br /> | rowspan="1"|2017/05/22<br /> |Aodong Li || 14:00 || 22:00 || 8 || <br /> *double-decoder without joint loss generalizes very bad<br /> *i'm trying double-decoder model with joint loss<br /> |-<br /> | rowspan="1"|2017/05/23<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *details about experiment 1: <br /> hidden_size = 700<br /> emb_size = 510<br /> learning_rate = 0.0005 (0.001 in prior)<br /> small data, <br /> 2nd translator uses as training data the concat(Chinese, machine translated English), <br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.19'''<br /> Overfitting? In overall, the 2nd translator performs worse than baseline<br /> *details about experiment 2: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> double-decoder model with joint loss which means the final loss = 1st decoder's loss + 2nd <br /> decoder's loss<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''39.04'''<br /> The 1st decoder's output is generally better than 2nd decoder's output. The reason may be that <br /> the second decoder only learns from the first decoder's hidden states because their states are <br /> almost the same.<br /> *DISCOVERY: <br /> The reason why double-decoder without joint loss generalizes very bad is that the gap between<br /> force teaching mechanism (training process) and beam search mechanism (decoding process)<br /> propagates and expands the error to the output end, which destroys the model when decoding.<br /> *next:<br /> Try to train double-decoder model without joint loss but with beam search on 1st decoder.<br /> |-<br /> | rowspan="1"|2017/05/24<br /> |Aodong Li || 13:00 || 21:30 || 8 || <br /> *code double-attention one-decoder model<br /> *code double-decoder model<br /> |-<br /> <br /> | rowspan="1"|2017/05/24<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> <br /> | rowspan="2"|2017/05/25<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *write document of tf_translate project <br /> *read neural machine translation paper <br /> *read tf_translate code <br /> |-<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * code and debug double attention model<br /> |-<br /> <br /> | rowspan="1"|2017/05/27<br /> |Shipan Ren || 9:30 || 18:30 || 9 || <br /> *read tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> | rowspan="1"|2017/05/28<br /> |Aodong Li || 15:00 || 22:00 || 7 || <br /> *details about experiment: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> when decoding:<br /> final_attn = attn_1 + attn_2 best result of our model: '''43.50'''<br /> final_attn = 2/3attn_1 + 4/3attn_2 best result of our model: '''41.22'''<br /> final_attn = 4/3attn_1 + 2/3attn_2 best result of our model: '''43.58'''<br /> |-<br /> | rowspan="1"|2017/05/30<br /> |Aodong Li || 15:00 || 21:00 || 6 || <br /> *details about experiment 1: <br /> hidden_size = 500<br /> emb_size = 310<br /> learning_rate = 0.001<br /> small data, <br /> 2nd translator uses as training data both Chinese and machine translated English<br /> Chinese and English use different encoders and different attention<br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''random initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''42.36'''<br /> * details about experiment 2: <br /> '''final_attn = 2/3attn_1 + 4/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.32'''<br /> * details about experiment 3: <br /> '''final_attn = attn_1 + attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.41''' and it seems more stable<br /> |-<br /> | rowspan="2"|2017/05/31<br /> |Shipan Ren || 10:00 || 19:30 || 9.5 || <br /> *run and test tf_translate code <br /> *write document of tf_translate project <br /> |-<br /> |Aodong Li || 12:00 || 20:30 || 8.5 || <br /> * details about experiment 1: <br /> '''final_attn = 4/3attn_1 + 2/3attn_2'''<br /> 2nd translator uses '''constant initialized embedding'''<br /> *results (BLEU): <br /> BASELINE: 43.87<br /> best result of our model: '''45.79'''<br /> * That only make English word embedding at encoder constant and train all the other embedding and parameters achieves an even higher bleu score 45.98 and the results are stable.<br /> * The quality of English embedding at encoder plays an pivotal role in this model.<br /> * Preparation of big data. <br /> |-<br /> | rowspan="1"|2017/06/01<br /> |Aodong Li || 13:00 || 24:00 || 11 || <br /> * Only make the English encoder's embedding constant -- 45.98<br /> * Only initialize the English encoder's embedding and then finetune it -- 46.06<br /> * Share the attention mechanism and then directly add them -- 46.20<br /> * Run double-attention model on large data<br /> |-<br /> | rowspan="1"|2017/06/02<br /> |Aodong Li || 13:00 || 22:00 || 9 || <br /> * Baseline bleu on large data is 30.83 with '''30000''' output vocab<br /> * Our best result is 31.53 with '''20000''' output vocab<br /> |-<br /> | rowspan="1"|2017/06/03<br /> |Aodong Li || 13:00 || 21:00 || 8 || <br /> * Train the model with 40 batch size and with concat(attn_1, attn_2)<br /> * the best result of model with 40 batch size and with add(attn_1, attn_2) is 30.52<br /> |-<br /> | rowspan="1"|2017/06/05<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/06<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/07<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/08<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/09<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/12<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/13<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/14<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> |-<br /> | rowspan="1"|2017/06/15<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/16<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Prepare for APSIPA paper<br /> * Read paper about MT involving grammar<br /> |-<br /> | rowspan="1"|2017/06/19<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Completed APSIPA paper<br /> * Took new task in style translation<br /> |-<br /> | rowspan="1"|2017/06/20<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried synonyms substitution<br /> |-<br /> | rowspan="1"|2017/06/21<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried post edit like synonyms substitution but this didn't work<br /> |-<br /> | rowspan="1"|2017/06/22<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> |-<br /> | rowspan="2"|2017/06/23<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read neural machine translation paper <br /> * read and run tf_translate code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained a GRU language model to determine similar word<br /> * This didn't work because semantics is not captured<br /> |-<br /> | rowspan="2"|2017/06/26<br /> |Shipan Ren || 10:00 || 21:00 || 11 || <br /> * read paper：LSTM Neural Networks for Language Modeling<br /> * read and run ViVi_NMT code <br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Tried to figure out new ways to change the text style<br /> |-<br /> | rowspan="2"|2017/06/27<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Trained seq2seq model to solve this problem<br /> * Semantics are stored in fixed-length vectors by a encoder and a decoder generate sequences on this vector<br /> |-<br /> | rowspan="2"|2017/06/28<br /> |Shipan Ren || 10:00 || 19:00 || 9 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * installed tensorflow0.1 and tensorflow1.0 on my pc and debugged ViVi_NMT<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Cross-domain seq2seq w/o attention and w/ attention models didn't work because of overfitting<br /> |-<br /> | rowspan="2"|2017/06/29<br /> |Shipan Ren || 10:00 || 20:00 || 10 || <br /> * read the API of tensorflow<br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="2"|2017/06/30<br /> |Shipan Ren || 10:00 || 24:00 || 14 || <br /> * debugged ViVi_NMT and tried to upgrade code version to tensorflow1.0 (on server)<br /> * accomplished this task <br /> * found the new version saves more time，has lower complexity and better bleu than before<br /> |-<br /> |Aodong Li || 10:00 || 19:00 || 8 || <br /> * Read style transfer papers<br /> |-<br /> | rowspan="1"|2017/07/03<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on small data sets (Chinese-English)<br /> * tested these checkpoint<br /> <br /> |-<br /> | rowspan="1"|2017/07/04<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * recorded experimental results<br /> * found version 1.0 of the code save more training time, has less complexity and these two version of the code has a similar Bleu value<br /> * found that the Bleu is still good when the model is over fitting<br /> * reason: the test set and training set are similar in content and style on small data set<br /> <br /> |-<br /> | rowspan="1"|2017/07/05<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * run two versions of the code on big data sets (Chinese-English)<br /> * read NMT papers<br /> <br /> |-<br /> | rowspan="1"|2017/07/06<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * out of memory（OOM） error occurred when version 0.1 of code was trained using large data set，but version 1.0 worked<br /> * reason: improper distribution of resources by the tensorflow0.1 version leads to exhaustion of memory resources<br /> * I've tried many times, and version 0.1 worked<br /> |-<br /> | rowspan="1"|2017/07/07<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * tested these checkpoints and recorded experimental results<br /> * the version 1.0 code saved 0.06 second per step than the version 0.1 code<br /> |-<br /> | rowspan="1"|2017/07/08<br /> |Shipan Ren || 9:00 || 21:00 || 12 || <br /> * downloaded the wmt2014 data set<br /> * used the English-French data set to run the code and found the translation is not good<br /> * reason:no data preprocessing is done<br /> <br /> |-<br /> | rowspan="1"|2017/07/10<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * run two versions of the code on small data sets (Chinese-English) and tested these checkpoint<br /> <br /> <br /> |-<br /> | rowspan="1"|2017/07/11<br /> |Shipan Ren || 9:00 || 20:00 || 11 || <br /> * <br /> <br /> |-<br /> <br /> | rowspan="1"|2017/07/18<br /> |Jiayu Guo || 8:30|| 22:00 || 14 ||<br /> * read model code.<br /> |-<br /> | rowspan="1"|2017/07/19<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of bleu.<br /> |-<br /> | rowspan="1"|2017/07/20<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read papers of attention mechanism.<br /> |-<br /> <br /> | rowspan="1"|2017/07/21<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/24<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * read model code.<br /> |-<br /> <br /> | rowspan="1"|2017/07/25<br /> |Jiayu Guo || 9:00|| 23:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/26<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/27<br /> |Jiayu Guo || 10:00|| 24:00 || 14 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/07/28<br /> |Jiayu Guo || 9:00|| 24:00 || 15 ||<br /> * process document<br /> <br /> |<br /> |-<br /> | rowspan="1"|2017/07/31<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * split ancient language text to single word<br /> |<br /> |-<br /> | rowspan="1"|2017/08/1<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run seq2seq_model<br /> |<br /> |-<br /> | rowspan="1"|2017/08/2<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/3<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/4<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * search new data(Songshu)<br /> |-<br /> | rowspan="1"|2017/08/7<br /> |Jiayu Guo || 9:00|| 22:00 || 13 ||<br /> * process document<br /> |-<br /> | rowspan="1"|2017/08/8<br /> |Jiayu Guo || 10:00|| 21:00 || 11 ||<br /> * read tensorflow <br /> |-<br /> | rowspan="1"|2017/08/9<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * run model with the data of which ancient content was split by single character.<br /> |-<br /> <br /> | rowspan="1"|2017/08/10<br /> |Jiayu Guo || 9:00|| 23:00 || 13 ||<br /> * process data of Songshu<br /> * read papers of CNN <br /> |-<br /> | rowspan="1"|2017/08/11<br /> |Jiayu Guo || 10:00|| 23:00 || 13 ||<br /> * learn about Graphic Model of LSTM-Projected BPTT<br /> * search for data available for translation (Twenty-four-Shi)<br /> |-s<br /> | rowspan="1"|2017/08/12<br /> |Jiayu Guo || 11:00|| 23:30 || 12 ||<br /> * run model with data including Shiji、Zizhitongjian.<br /> |-<br /> | rowspan="1"|2017/08/13<br /> |Jiayu Guo || 13:00|| || ||<br /> * test results.<br /> checkpoint-100000 translation model<br /> BLEU： 11.11<br /> <br /> *source:在秦者名错，与张仪争论,於是惠王使错将伐蜀，遂拔，因而守之。<br /> *target:在秦国的名叫司马错，曾与张仪发生争论，秦惠王采纳了他的意见，于是司马错率军攻蜀国，攻取后，又让他做了蜀地郡守。<br /> *trans：当时秦国的人都很欣赏他的建议，与张仪一起商议，所以吴王派使者率军攻打蜀地，一举攻，接着又下令守城。<br /> *source:神大用则竭，形大劳则敝，形神离则死。 <br /> *target:精神过度使用就会衰竭，形体过度劳累就会疲惫，神形分离就会死亡。 <br /> *trans: 精神过度就可衰竭,身体过度劳累就会疲惫，地形也就会死。<br /> *source:今天子接千岁之统，封泰山，而余不得从行，是命也夫，命也夫！<br /> *target:现天子继承汉朝千年一统的大业，在泰山举行封禅典礼而我不能随行，这是命啊，是命啊！ <br /> *trans: 现在天子可以继承帝位的成就爵位，爵位至泰山，而我却未能执行先帝的命运。<br /> <br /> *1.data used Zizhitongjian only(6,000 pairs), we can get BLEU 6 at most.<br /> *2.data used Zizhitongjian only(12,000 pairs), we can get BLEU 7 at most.<br /> *3.data used Shiji and Zizhitongjian(43,0000 pairs), we can get BLEU about 9.<br /> *4.data used Shiji and Zizhitongjian(43,0000 pairs), and split the ancient language text one character by one, we can get BLEU 11.11 at most.<br /> *The main factors now is the data(including pairs of sentence、the quality——cause the modern language text include context information.<br /> <br /> |-<br /> |-<br /> | rowspan="1"|2017/08/19<br /> |Jiayu Guo || 13:00|| 23:00 || 10 ||<br /> * read source code.<br /> |-<br /> |-<br /> | rowspan="1"|2017/08/20<br /> |Jiayu Guo || 13:00|| 22:00 || 9 ||<br /> * read source code.<br /> |-<br /> }<br /> <br /> ===Time Off Table===<br /> <br /> {| class="wikitable"<br /> ! Date !! Yang Feng !! Jiyuan Zhang <br /> |-<br /> |}<br /> <br /> ==Past progress==<br /> [[nlp-progress 2017/03]]<br /> <br /> [[nlp-progress 2017/02]]<br /> <br /> [[nlp-progress 2017/01]]<br /> <br /> [[nlp-progress 2016/12]]<br /> <br /> [[nlp-progress 2016/11]]<br /> <br /> [[nlp-progress 2016/10]]<br /> <br /> [[nlp-progress 2016/09]]<br /> <br /> [[nlp-progress 2016/08]]<br /> <br /> [[nlp-progress 2016/05/01 -- 08/16 | nlp-progress 2016/05-07]]<br /> <br /> [[nlp-progress 2016/04]]</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-14 NLP Status Report 2017-8-14 2017-08-21T00:53:42Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *polished the couplet model <br /> || <br /> Code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * organized the results of the experiment<br /> * learned how to use THUMT and how did it work<br /> ||<br /> * train translation models by using THUMT<br /> * test the bleu of these models<br /> * compare with our system<br /> |-<br /> <br /> |Jiayu Guo||<br /> <br /> ||<br /> <br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-7-31 NLP Status Report 2017-7-31 2017-08-21T00:51:08Z

<p>Renshipan：</p> <hr /> <div><br /> {| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/31<br /> |Jiyuan Zhang ||<br /> *made the poster for ACL [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/9/95/Acl2017-poster.pdf]<br /> *attempted to fix repeated word, but failed<br /> *done some work of n-gram model of the couplet<br /> || <br /> *generate streame according to a couplet<br /> *complete the task of filling in the blanks of a couplet<br /> <br /> |-<br /> |Aodong LI ||<br /> * Got 55,000+ Englsih poems and 260,000+ lines after preprocessing<br /> * Added phase separators as the style indicator, and every line has at least one separator<br /> * Training loss didn't decrease very much, only from 440 to 50<br /> * The translation quality deteriorated when added language model<br /> ||<br /> * Try to use a larger language model to decrease the training loss<br /> * Try to use character-based MT in English-Chinese translation<br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * looked for the performance(the bleu value) of other models <br /> on the WMT2014 dataset from the published papers,but not found.<br /> * installed and built Moses on the server <br /> ||<br /> * train statistical machine translation model and test it <br /> toolkit: Moses<br /> data sets:WMT2014 en-de、en-fr data sets<br /> * collate experimental results.compare our baseline model with Moses <br /> |-<br /> <br /> |Jiayu Guo||<br /> *process document.Until now, Shiji has been split up to 2,4000 pairs of sentence.<br /> *Zizhitongjian has been split up to 1,6000 pairs.<br /> ||<br /> *adjust jieba source code, in order to make jieba more accurate for ancient language wordpiece<br /> *read model source code<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-7-24 NLP Status Report 2017-7-24 2017-08-21T00:50:49Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/24<br /> |Jiyuan Zhang ||<br /> *<br /> || <br /> *make the poster for ACL<br /> *complete neural model for the couplet<br /> |-<br /> |Aodong LI ||<br /> * Completed the shallow fusion of news-domain translation with dialog-domain style.<br /> * The style was not obvious since the dialog dataset has no specific style indicator.<br /> * Some examples:<br /> 全程预计 00 天 , 团费大约 0.0万元人民币。<br /> w/ style: it is estimated that 00 days of the entire project will be about 00 million yuan .<br /> w/o style: the whole world is expected to be about 00 days , with a total of 00,000 yuan rmb .<br /> <br /> 在美国九一一恐怖攻击周年左右 , 东南亚各地的西方外交使节团纷纷关闭 , 因为 <br /> 它们遭到与欧萨玛 . 宾拉登的盖达组织及其地方联盟有关的威胁。<br /> w/ style: on the anniversary of the sept 0 terrorist attack , the western dpp diplomatic <br /> envoys in southeast asia were shut down because they were with the threat to al qaeda<br /> bin laden and al - qaeda 's relevant alliance .<br /> w/o style: on the anniversary of the sept 0 terrorist attack , the western dpp diplomatic envoys<br /> in southeast asia were shut off because they were closely connected with osama bin laden 's al <br /> qaeda and al - qaeda 's relevant alliances .<br /> ||<br /> * Find the dataset with obvious style indicators.<br /> * Try to quantify the result to determine if it is effective.<br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * trained two models of the baseline using WMT2014 en-fr datasets<br /> under training <br /> new version saved more time<br /> <br /> * read some papers（memory-augmented-nmt and Memory augmented Chinese-Uyghur Neural Machine Translation） <br /> ||<br /> * read memory-augmented-nmt code<br /> * read papers about memory augmented NMT <br /> |-<br /> <br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-7-17 NLP Status Report 2017-7-17 2017-08-21T00:50:33Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/17<br /> |Jiyuan Zhang ||<br /> *<br /> || <br /> *generate streame according to a couplet <br /> *try my best to complete the task of filling in the blanks of a couplet<br /> |-<br /> |Aodong LI ||<br /> *<br /> <br /> ||<br /> *<br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * found ways to tokenize the WMT2014 data <br /> rewrote prepare_data.py form moses-smt<br /> used the tokenizer of moses-smt<br /> <br /> *train two versions of the code on WMT2014 en-de and en-fr datasets<br /> tested these checkpoints of en-de dataset<br /> <br /> ||<br /> * tested these checkpoints of en-fr dataset<br /> * record the result and do analysis <br /> * read papers about memory augmented NMT <br /> |-<br /> <br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-7 NLP Status Report 2017-8-7 2017-08-21T00:48:52Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/7<br /> |Jiyuan Zhang ||<br /> *generated streame according to a couplet<br /> *almost completed the task of filling in the blanks of a couplet<br /> || <br /> *continue to perfect the couplet model<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * train statistical machine translation model and test it <br /> toolkit: Moses<br /> data sets:WMT2014 en-de、en-fr data sets<br /> * collate experimental results.compare our baseline model with Moses <br /> en-de dataset<br /> Moses:15.4<br /> Baseline:14.87<br /> <br /> en-fr datasets<br /> under training<br /> ||<br /> * read memory-augment NMT code <br /> * think about the next step work<br /> |-<br /> <br /> |Jiayu Guo||<br /> *process document.Until now, Shiji has been split up to 2,4000 pairs of sentence.<br /> *Zizhitongjian has been split up to 1,6000 pairs.<br /> ||<br /> *adjust jieba source code, in order to make jieba more accurate for ancient language wordpiece<br /> *read model source code<br /> |-<br /> ||<br /> ||<br /> *run Shiji-based model<br /> *combine Shiji and Zizhitongjian,then run the model<br /> 在秦国做国人的第二，与张仪一起进言，于是派司马错率兵攻打蜀地，于是攻克了，乘势攻下了。<br /> 因冯直有的言辞和他的情况，往往得不到安宁罢了。<br /> 　　到了道术 _UNK 的方法，弃 _UNK _UNK ， _UNK _UNK ，赦免天时，使人著儒家的态度。<br /> 　　教化礼制 _UNK 法度，一定要根据这方法发展治理百姓。<br /> 　　无德行的人，无不分明，所以能够忍受八万余人。<br /> 　　 _UNK _UNK 就完备了，那么国家的强盛就会疲惫， _UNK 鬼神就会<br /> 　　当今天子到各封地，建立到长安，而国内没有得到右翼，就当上了出使最下等的事。我死了，你必然当上了大官；不为我要死呢。<br /> 　　孔子说： ‘ 我想把她作俸禄的恶名，不在于见到他的行事，就感动得很容易考虑。<br /> 　　他的部属都认为好人很好，认为他不了解他的大义，被处以死刑的刑罚，不敢再推辞。<br /> 汉王朝兴起，到达天子，平定华夏 _UNK ，消除 _UNK ，消除 _UNK ，整治 _UNK ，统一度量衡，整治满足本人的需要，准备调节 _UNK ，请举行 _UNK 。<br /> 　　韩厥慈爱孝顺的功绩， _UNK _UNK 为他。<br /> 作《项羽本纪》第二十七。<br /> 　　《春秋》以后，诸侯独断专行，安抚四方国家； _UNK 到秦国，最终并吞夏朝的土地，消灭了周室，他的封号。<br /> 　　作《魏公子 _UNK 列传》 _UNK 。<br /> 太公、孙子、吴、王子继位则有明智而有明智之心，违背天道，无不分明。<br /> 　　运筹帷幄之中，安抚耕种；他的运行则驰骋弋猎的声音，招来了一套 _UNK 的利益，违背风俗，补救积弊，去掉节俭的性能。<br /> 　　成王年龄尚幼，公室非常怀疑他，淮夷背叛他，于是召公振兴了成王，占有了天下的政权。<br /> }</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-14 NLP Status Report 2017-8-14 2017-08-21T00:48:04Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/8/14<br /> |Jiyuan Zhang ||<br /> *polished the couplet model <br /> || <br /> Code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * organized the results of the experiment<br /> * learned how to use THUMT and how did it work<br /> ||<br /> * train translation models by using THUMT<br /> * test the bleu of these models<br /> * compared with our system<br /> |-<br /> <br /> |Jiayu Guo||<br /> <br /> ||<br /> <br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-14 NLP Status Report 2017-8-14 2017-08-21T00:36:22Z

<p>Renshipan：</p> <hr /> <div><br /> <br /> {| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/3<br /> |Jiyuan Zhang ||<br /> *polished the couplet model <br /> || <br /> Code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> <br /> ||<br /> <br /> |-<br /> <br /> |Jiayu Guo||<br /> <br /> ||<br /> <br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-14 NLP Status Report 2017-8-14 2017-08-21T00:35:58Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/3<br /> |Jiyuan Zhang ||<br /> *polished the couplet model <br /> ||<br /> Code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> ||<br /> |-<br /> |Shiyue Zhang || <br /> ||<br /> |-<br /> |Shipan Ren ||<br /> ||<br /> |-<br /> |Jiayu Guo||<br /> ||<br /> |-<br /> <br /> }<br /> <br /> <br /> <br /> <br /> <br /> {| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/3<br /> |Jiyuan Zhang ||<br /> *polished the couplet model <br /> || <br /> Code refactoring for poem system<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> <br /> ||<br /> <br /> |-<br /> <br /> |Jiayu Guo||<br /> <br /> ||<br /> <br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-7-10 NLP Status Report 2017-7-10 2017-08-21T00:31:04Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/10<br /> |Jiyuan Zhang ||<br /> *reproduced the couplet model using moses<br /> || <br /> *continue to modify the couplet<br /> |-<br /> |Aodong LI ||<br /> * Tried a seq2seq with style code model but it didn't work.<br /> * Coded attention-based seq2seq NMT in shallow fusion with a language model.<br /> ||<br /> * Complete coding and have a try. <br /> * Find more monolingual corpus and upgrade the model.<br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * run two versions of the code on small data sets (Chinese-English) and tested these checkpoint<br /> found version 1.0 save time about 0.03s per step, <br /> and these two version has similar complexity and bleu values <br /> found that the bleu is still good when the model is over fitting .<br /> (reason: the test set and the train set of small data set are similar in content and style) <br /> * run two versions of the code on big data sets (Chinese-English) . <br /> OOM（Out Of Memory） error occurred when version 0.1 was trained using large data set，but version 1.0 worked <br /> reason: improper distribution of resources by the tensorflow0.1 frame leads to exhaustion of memory resources <br /> I had tried 4 times （just enter the same command）, and version 0.1 worked <br /> found version 1.0 save time about 0.06s per step, and these two version has similar complexity and bleu values <br /> * downloaded the wmt2014 data set ,used the English-French data set to run the code and <br /> found the translation is not good (reason:improper word segmentation)<br /> ||<br /> * do word segmentation on wmt2014 data set <br /> * run two versions of the code on wmt2014 data set <br /> * record the result and do analysis <br /> * learn and train moses(use big data sets (Chinese-English))<br /> |-<br /> <br /> <br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-8-7 NLP Status Report 2017-8-7 2017-08-07T05:08:16Z

<p>Renshipan：</p> <hr /> <div>{| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/3<br /> |Jiyuan Zhang ||<br /> *generated streame according to a couplet<br /> *almost completed the task of filling in the blanks of a couplet<br /> || <br /> *continue to perfect the couplet model<br /> |-<br /> |Aodong LI ||<br /> <br /> ||<br /> <br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * train statistical machine translation model and test it <br /> toolkit: Moses<br /> data sets:WMT2014 en-de、en-fr data sets<br /> * collate experimental results.compare our baseline model with Moses <br /> en-de dataset<br /> Moses:15.4<br /> Baseline:14.87<br /> <br /> en-fr datasets<br /> under training<br /> ||<br /> * read memory-augment NMT code <br /> * think about the next step work<br /> |-<br /> <br /> |Jiayu Guo||<br /> *process document.Until now, Shiji has been split up to 2,4000 pairs of sentence.<br /> *Zizhitongjian has been split up to 1,6000 pairs.<br /> ||<br /> *adjust jieba source code, in order to make jieba more accurate for ancient language wordpiece<br /> *read model source code<br /> |-<br /> |}</div>

Renshipan http://index.cslt.org/mediawiki/index.php/NLP_Status_Report_2017-7-31 NLP Status Report 2017-7-31 2017-07-31T04:57:55Z

<p>Renshipan：</p> <hr /> <div><br /> {| class="wikitable"<br /> !Date !! People !! Last Week !! This Week<br /> |-<br /> | rowspan="6"|2017/7/3<br /> |Jiyuan Zhang ||<br /> *made the poster for ACL<br /> *attempted to fix repeated word, but failed<br /> *done some work of n-gram model of the couplet<br /> || <br /> *generate streame according to a couplet<br /> *complete the task of filling in the blanks of a couplet<br /> <br /> |-<br /> |Aodong LI ||<br /> * Got 55,000+ Englsih poems and 260,000+ lines after preprocessing<br /> * Added phase separators as the style indicator, and every line has at least one separator<br /> * Training loss didn't decrease very much, only from 440 to 50<br /> * The translation quality deteriorated when added language model<br /> ||<br /> * Try to use a larger language model to decrease the training loss<br /> * Try to use character-based MT in English-Chinese translation<br /> |-<br /> |Shiyue Zhang || <br /> <br /> ||<br /> <br /> |-<br /> |Shipan Ren ||<br /> * looked for the performance(the bleu value) of other models <br /> on the WMT2014 dataset from the published papers,but not found.<br /> * installed and built Moses on the server <br /> ||<br /> * train statistical machine translation model and test it <br /> toolkit: Moses<br /> data sets:WMT2014 en-de、en-fr data sets<br /> * collate experimental results.compare our baseline model with Moses <br /> |-<br /> <br /> |Jiayu Guo||<br /> *process document.<br /> *Shiji has been split up to 2,5000 pairs of sentence.<br /> *Zizhitongjian has been split up to 2,0000 pairs.<br /> ||<br /> *adjust jieba source code<br /> |-<br /> |}</div>

Renshipan