2013-07-22 - 版本历史

Wangd：/* Experiments */

2013-07-22T12:38:06Z

‎Experiments

Wangd：/* Experiments */

2013-07-22T12:32:19Z

‎Experiments

Wangd：/* Experiments */

2013-07-22T12:31:34Z

‎Experiments

Wangd：/* Embedded progress */

2013-07-22T12:30:57Z

‎Embedded progress

2013年7月22日 (一) 12:29 Wangd

2013-07-22T12:29:14Z

Wangd：以内容“== Data sharing == * LM count files still undelivered! == DNN progress == === Experiments === * Sparse DNN. 1200-1200-1200-3536 1200-1200-1200-353...”创建新页面

2013-07-22T12:27:00Z

以内容“== Data sharing == * LM count files still undelivered! == DNN progress == === Experiments === * Sparse DNN. 1200-1200-1200-3536 1200-1200-1200-353...”创建新页面

新页面

== Data sharing ==

* LM count files still undelivered!

== DNN progress ==

=== Experiments ===
* Sparse DNN.

1200-1200-1200-3536 1200-1200-1200-3536-sparse0.3 (sparsity 1/5)
original atlas: RT 2.3 RT 2.3
atlas sparse: RT 54 RT 14
NIST smatmat: RT 27.3 RT 5.98

800-800-800-2108 800-800-800-2108-sparse0.3 (sparsity 2/5):
original atlas: RT 1.1 RT 1.1
NIST smatmat: RT 11.9 RT 5.5

Conclusions:
1. the atlas works well for both non-sparse and sparse.
2. sparsity does not work if the sparsity rate is low. It looks the sparsity computing can
outperform the non-sparsity computing only if the sparsity rate is higher than 1/15.
3. In another words, to employ sparsity, the cost that first should be taken is the error rate
increase with the 1/15 compression.
4. The sparse approach seems more useful for storage: if the sparsity is higher than 1/2, then the
storage of CSR/CSC will start to save storage.
5. Possibly unit-based sparsity instead of weight sparsity.

=== Tencent exps ===

=== GPU & CPU merge ===

#Hold

== Embedded progress ==

*Tested various PS models:

ID model feature WER RT storage

semi_10000 semi HMM s2-4x 6.30% 0.80 10.2M
semi_5000 semi HMM s2-4x 6.70% 0.74 5.2M
semi_5000 semi HMM 1c-d-dd 9.11% 0.91 1.3M
ptm_5000 PTM HMM s2-4x 6.47% 2.15 1.3M

So there is not a perfect which wins in terms all the criteria. Looks like semi-5000 is an acceptable trade-off.

@@ 第8行： / 第8行： @@
 * Sparse DNN.
-:1200-1200-1200-3536                 1200-1200-1200-3536-sparse0.3 (sparsity 1/5)
+<pre>
-:original atlas:  RT 2.3                         RT 2.3
+-800-800-2108                    800-800-800-2108-sparse0.3 (sparsity 2/5):
-:atlas sparse:    RT 54                          RT 14
+original atlas: RT 1.1                          RT 1.1
-:NIST smatmat:    RT 27.3                        RT 5.98
+NIST smatmat:   RT 11.9                         RT 5.5
+</pre>
-−
-:800-800-800-2108                    800-800-800-2108-sparse0.3 (sparsity 2/5):
-:original atlas: RT 1.1                          RT 1.1
-:NIST smatmat:   RT 11.9                         RT 5.5
 <pre>

@@ 第40行： / 第40行： @@
 *Tested various PS models:
-:ID            model        feature    WER       RT          storage
+<pre>
 ID            model        feature    WER       RT          storage
-:semi_10000    semi HMM    s2-4x    6.30%       0.80          10.2M
-:semi_5000     semi HMM    s2-4x    6.70%       0.74          5.2M
-:semi_5000     semi HMM    1c-d-dd  9.11%       0.91          1.3M
-:ptm_5000      PTM HMM     s2-4x    6.47%       2.15          1.3M
 So there is not a perfect which wins in terms all the criteria. Looks like semi-5000 is an acceptable trade-off.

@@ 第6行： / 第6行： @@
 === Experiments ===
-* Sparse DNN.
+* Sparse DNN on the ARM board
 <pre>

@@ 第19行： / 第19行： @@
 :NIST smatmat:   RT 11.9                         RT 5.5
 Conclusions:
-# the atlas works well for both non-sparse and sparse.
+. the atlas works well for both non-sparse and sparse.
-# sparsity does not work if the sparsity rate is low. It looks the sparsity computing can
+. sparsity does not work if the sparsity rate is low. It looks the sparsity computing can
 outperform the non-sparsity computing only if the sparsity rate is higher than 1/15.
-# In another words, to employ sparsity, the cost that first should be taken is the error rate
+. In another words, to employ sparsity, the cost that first should be taken is the error rate
 increase with the 1/15 compression.
-# The sparse approach seems more useful for storage: if the sparsity is higher than 1/2, then the
+. The sparse approach seems more useful for storage: if the sparsity is higher than 1/2, then the
 storage of CSR/CSC will start to save storage.
-# Possibly unit-based sparsity instead of weight sparsity.
+. Possibly unit-based sparsity instead of weight sparsity.
 === Tencent exps ===