“Gigabye LM”版本间的差异
第33行: | 第33行: | ||
2 1e-6 1e-6 1e-6 5M logprob= -9.26982e+06 ppl= 150.96 ppl1= 164.9 | 2 1e-6 1e-6 1e-6 5M logprob= -9.26982e+06 ppl= 150.96 ppl1= 164.9 | ||
3 1e-7 1e-6.5 1e-6.5 11M logprob= -9.09681e+06 ppl= 137.467 ppl1= 149.913 | 3 1e-7 1e-6.5 1e-6.5 11M logprob= -9.09681e+06 ppl= 137.467 ppl1= 149.913 | ||
+ | |||
+ | |||
+ | == 3. word-based 3-gram == | ||
+ | org th-7 th-6.5 th-6 | ||
+ | 10k: 52M 23M 8M 4M | ||
+ | 20k: 57M 24M 9M 4M |
2012年9月13日 (四) 05:06的版本
1. very initial, without any prunning, character based. Here is the size and perplexity.
The training is with Gigabytes except the cna data, and ppl testing is based on a sub set from the cna data (big52gb applied)
2gram:
25M 2gram.4000.gz: 0 zeroprobs, logprob= -9.39983e+06 ppl= 161.965 ppl1= 177.141
3gram:
47M 3gram.500.gz:0 zeroprobs, logprob= -6.34868e+06 ppl= 85.1361 ppl1= 94.2525
117M 3gram.1000.gz :0 zeroprobs, logprob= -7.43809e+06 ppl= 80.6408 ppl1= 87.7439
195M 3gram.2000.gz:0 zeroprobs, logprob= -7.95872e+06 ppl= 79.9875 ppl1= 86.5196
221M 3gram.3000.gz:0 zeroprobs, logprob= -8.04799e+06 ppl= 80.2418 ppl1= 86.7277
229M 3gram.4000.gz:0 zeroprobs, logprob= -8.15697e+06 ppl= 82.6585 ppl1= 89.3392
4gram:
205M 4gram.500.gz:0 zeroprobs, logprob= -6.25395e+06 ppl= 79.6739 ppl1= 88.0716
472M 4gram.1000.gz:0 zeroprobs, logprob= -7.21607e+06 ppl= 70.737 ppl1= 76.774
2. pruning the 4k 3gram LM.
Model 1gram 2gram 3gram size ppl 1 1e-7 1e-7 1e-7 30M logprob= -8.55982e+06 ppl= 102.796 ppl1= 111.532 2 1e-6 1e-6 1e-6 5M logprob= -9.26982e+06 ppl= 150.96 ppl1= 164.9 3 1e-7 1e-6.5 1e-6.5 11M logprob= -9.09681e+06 ppl= 137.467 ppl1= 149.913
3. word-based 3-gram
org th-7 th-6.5 th-6
10k: 52M 23M 8M 4M 20k: 57M 24M 9M 4M