2013-09-27 - 版本历史

Cslt：/* Continuous LM */

2013-09-29T15:32:22Z

‎Continuous LM

Cslt：/* Noisy training */

2013-09-29T15:29:27Z

‎Noisy training

Cslt：/* Noisy training */

2013-09-29T15:29:11Z

‎Noisy training

Cslt：/* FBank features */

2013-09-29T15:28:12Z

‎FBank features

Cslt：/* Sparse DNN */

2013-09-29T15:26:55Z

‎Sparse DNN

Cslt：/* Noisy training */

2013-09-29T15:24:57Z

‎Noisy training

Cslt：/* FBank features */

2013-09-29T15:24:06Z

‎FBank features

Cslt：/* FBank features */

2013-09-29T15:23:44Z

‎FBank features

Cslt：/* Sparse DNN */

2013-09-29T15:22:36Z

‎Sparse DNN

Cslt：/* Sparse DNN */

2013-09-29T15:22:17Z

‎Sparse DNN

@@ 第39行： / 第39行： @@
 . SogouQ n-gram building: 500M text data, 110k words. Two tests:
-  (1) using Tencent online1 and online2 transcription: online1 1651 online2: 1512
+  (1) using Tencent online1 and online2 transcription: online1: 1651 online2: 1512
   (2) using 70k sogouQ test set : ppl 33
-   This means the SogouQ text is significantly different from the online1 and online2 Tencent set, due to the highly different domain.
+   This means the SogouQ text is significantly different from the online1 and online2 Tencent set, due to the different domain.
 . NN LM
-    Using 11k words as input, 192 hidden layer. 500M text data from QA data. test with online2 transcription.
+    Using 11k words as the input, 192 units in the hidden layer. 500M text data from QA data. Test with online2 transcription.
-   (1)  Take 1-1024 from NN LM, and others predicted by 4-gram. n-gram baseline: 402.37; NN+ngram: 122.54
+   (1)  Predict the most frequent 1-1024 words with the NN LM, and others predicted by 4-gram. n-gram baseline: 402.37; NN+ngram: 122.54
-   (2)  Take 1-2048 from NN LM, and others predicted by 4-gram. n-gram baseline: 402.37; NN+ngram: 127.59
+   (2)  Predict the most frequent 1-2048 words with the NN LM, and others predicted by 4-gram. n-gram baseline: 402.37; NN+ngram: 127.59
-   (3)  Take 1024-2048 from NN LM, and others predicted by 4-gram. n-gram baseline: 402.37; NN+ngram: 118.92
+   (3)  Predict the most frequent 1024-2048 words with the  NN LM, and others predicted by 4-gram. n-gram baseline: 402.37; NN+ngram: 118.92
-Conclusions:  NN  LM is extremely good than n-gram, due to its smooth capacity.
+Conclusions:  NN  LM is extremely good than n-gram, due to its smooth capacity.It seems it helps more for the not-very-frequent words, which verifies its capability in smoothing.

@@ 第32行： / 第32行： @@
 The conclusions is that:
-. by sampling noises, most of the noise patterns can be learned efficiently and thus improve performance on noisy test data.
+# by sampling noises, most of the noise patterns can be learned efficiently and thus improve performance on noisy test data.
-. by sampling noises with high variance, performance on clean speech is largely remained.
+# by sampling noises with high variance, performance on clean speech is largely remained.
 ==Continuous LM ==

@@ 第26行： / 第26行： @@
 Sample noise segments randomly for each utterance. Using Dirichlet to sample noise distribution on various types, and use Gaussian to sample SNR.
-White noise with car noise are 1/3 respectively in the base distribution. The performance report is here:
+The first initial test involves white noise and  car noise are 1/3 respectively. The performance report is here:
 [http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:Chart3.png click here]

@@ 第14行： / 第14行： @@
 === FBank features ===
-hour testing: [http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/%E6%96%87%E4%BB%B6:Chart2.png  here]
+hour testing is done. The performance is significantly better than the MFCC. And the iteration 14 is better than the final iteration. This may be attributed to some over-fitting.
 === Tencent exps ===

@@ 第8行： / 第8行： @@
 * Optimal Brain Damage based sparsity is on going. Prepare the algorithm.
-* An interesting investigation is drop-out 50% weights after each iteration, and then re-training without sticky.
+* An interesting investigation is drop-out 50% weights after each iteration, and then re-training without sticky. The performance is a bit better than the original best. This might be attributed to some noisy turbulence that provides some change out of local minimum.
 Report on [http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/文件:Chart1.png here]

@@ 第10行： / 第10行： @@
 * An interesting investigation is drop-out 50% weights after each iteration, and then re-training without sticky.
-Report on [[文件:Chart1.png]]
+Report on [http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/文件:Chart1.png]
 === FBank features ===