“2013-06-21”版本间的差异

2013年6月21日 (五) 02:32的版本

Data sharing

• LM count files still undelivered!

DNN progress

Experiments

• sparse DNN: sticky training (retrain the nnet while keeping the sparsness)

zero small values(test set: 1900), with extremely sparseness:

Conclusion: The extremely sparse network can largely pertain the performance of DNN. The structure seems more important than the parameter tuning based on the structure.

• fixed-point DNN forwarding

1. Migrating the Atlas lib to ARM. Done.
2. Atlas/mkl independent implementation. Done. The non Atlas/MKL solution is much slower, but with sparse DNN, the difference is decreased.
3. Working on comparison of Atlas and non-Atlas on ARM.

Tencent exps

1. ：本月DNN方面，一开始大规模数据（6000小时）训练工作，现得到迭代7次结果如下：

Old Baseline New Baeline DNN-1000小时 DNN-6000小时

1900		8.4		6.8		4.3		3.9
2044		22.4		15.7		12.7		10.7
online1	35.6		32.7		25.8		24.6
online2	29.6		27.3		22.1		21.1
map		24.5		15.8		13.4		8.7
general	36		25.1		19.3		15.9

1. ：DNN+序列化dt，未使用Kaldi代码，重写中，接近完成，鉴于其中包含的各种trick，得到有效结果的时间不好预估。
2. ：调研二阶优化方式，LBFGS，Hessian Free算法，以及异步SGD算法，实现多机并行化实现。
3. ：调研RNN相关的Long Short-Term Memery算法在ASR实现。

GPU & CPU merge

1. Hold

RNN LM progress

• ????

Embedded progress

• Status:

ARM debug system is ready. Native compiling is possible.

PS system: SGE-based large scale training is ready.

Kaldi system: ARM migration is done.

Semi continuous model based on the Tencent 400h data is done.

• To be done

1. sparse DNN based engine Kaldi engine
2. sparse DNN based PS engine