2013-06-21

Data sharing

• LM count files still undelivered!

DNN progress

Experiments

• sparse DNN: sticky training (retrain the nnet while keeping the sparsness)

zero small values(test set: 1900), with extremely sparseness:

threshold	0	0.2	0.3	0.4	0.5
shrinkage%	0.0	66.4	81.6	0.90	0.94
without sticky: WER	7.55	9.46	53.23	98.99	-
with sticky: WER	7.55	7.56	7.87	8.81	9.87

Conclusion: The extremely sparse network can largely pertain the performance of DNN. The structure seems more important than the parameter tuning based on the structure.

• fixed-point DNN forwarding

1. Migrating the Atlas lib to ARM. Done.
2. Atlas/mkl independent implementation. Done. The non Atlas/MKL solution is much slower, but with sparse DNN, the difference is decreased.
3. Working on comparison of Atlas and non-Atlas on ARM.

GPU & CPU merge

1. Hold

RNN LM progress

• ????

Embedded progress

• Status:

ARM debug system is ready. Native compiling is possible.

PS system: SGE-based large scale training is ready.

Kaldi system: ARM migration is done.

Semi continuous model based on the Tencent 400h data is done.

Sys	WER	RT(on server)	RT (on arm)
SP model	8.81	0.07
Tencent(cnt.)	5.04	0.31
Tencent(semi.)	-	-

• To be done

1. sparse DNN based engine Kaldi engine
2. sparse DNN based PS engine