cslt Wiki - 用户贡献 [zh-cn]

Sheng Su 2015-10-26

2015-10-26T23:05:41Z

Susheng：以“Last week: * Four-GPU training done. The result performs good. I will let Mengyuan put the result to CVSS. This week: * Do some optimization, improve stability.”为内容创建页面

Last week:
* Four-GPU training done. The result performs good. I will let Mengyuan put the result to CVSS.

This week:
* Do some optimization, improve stability.

2015-10-26

2015-10-26T23:01:06Z

Susheng：

[[Mengyuan Zhao 2015-10-26]]

[[Rong Liu 2015-10-26]]

[[Yang Yu 2015-10-26]]

[[Hui Tang 2015-10-26]]

[[Xiangyu Zeng 2015-10-26]]

[[Chao Xing]]

[[Tianyi Luo 2015-10-26]]

[[Lantian Li 2015-10-26]]

[[Zhiyuan Tang 2015-10-26]]

[[Sheng Su 2015-10-26]]

Sheng Su 2015-10-19

2015-10-20T00:49:58Z

Susheng：以“Last week: * I have find out the reason why two GPU work well, and four GPU don't. This base on two facts: *- 1. Mini-batch training : sum the gradient of all the...”为内容创建页面

Last week:
* I have find out the reason why two GPU work well, and four GPU don't. This base on two facts:
*- 1. Mini-batch training : sum the gradient of all the frames in the batch
*- 2. Mini-batch size : the baseline will not converge if we set mini-batch size over 1024
* Reason:
*- Mini-batch size is M, after N mini-batch we sum all the gradient from 4 GPU and update the net once.(during the N mini-batch, we don’t update the net).This is equal to the baseline with mini-batch size of M*N*4, much larger than baseline. However if we update the net during the N mini-batch, it seems like, to some extent, reduce the mini-batch size. That’s why two GPUs work well, and four GPUs don’t.
This week:
* 1. Try net average.
* 2. Learning NG-SGD.

2015-10-19

2015-10-20T00:42:14Z

Susheng：

[[Mengyuan Zhao 2015-10-19]]

[[Xiangyu Zeng 2015-10-19]]

[[Tianyi Luo 2015-10-19]]

[[Dongxu Zhang 2015-10-19]]

[[Xiaoxi Wang 2015-10-19]]

[[Chao Xing 2015-10-19]]

[[Yang Yu 2015-10-19]]

[[Zhiyong Zhang 2015-10-19]]

[[Xuewei Zhang 2015-10-19]]

[[Hui Tang 2015-10-19]]

[[Rong Liu 2015-10-19]]

[[Zhiyuan Tang 2015-10-19]]

[[Sheng Su 2015-10-19]]

Sheng Su 2015-10-12

2015-10-12T12:07:29Z

Susheng：以“four GPU training: -- * having tried to change learning rate, mini-batch size and the gap, still diverge. * having tried to use asynchronous way to update, still div...”为内容创建页面

four GPU training:
--
* having tried to change learning rate, mini-batch size and the gap, still diverge.
* having tried to use asynchronous way to update, still diverge.
* keep going to find the reason of divergency, and going to use some other methods to try.

Asr-intern-week

2015-09-01T02:59:13Z

Susheng：

==Intern Working Timetable==

{|class=wikitable
! !!周一 !! 周二 !! 周三 !! 周四 !! 周五 !! 周六 !! 周日
|-
|Dong Xu Zhang|| X ||X ||X ||? ||X ||? ||X
|-
|}

{|class=wikitable
! !!周一 !! 周二 !! 周三 !! 周四 !! 周五 !! 周六 !! 周日
|-
|Sheng Su|| X ||X ||X ||X ||X ||X ||X
|-
|}

Sheng Su 2015-08-17

2015-08-17T11:15:49Z

Susheng：以“Last week : Multi-GPU program, success to speed up. This week : Test the program, in order to get a good result.”为内容创建页面

Last week :
Multi-GPU program, success to speed up.

This week :
Test the program, in order to get a good result.

2015-08-17

2015-08-17T11:09:17Z

Susheng：

[[Tianyi Luo 2015-08-17]]

[[Mengyuan Zhao 2015-08-17]]

[[Chao Xing 2015-08-17]]

[[Sheng Su 2015-08-17]]

Weekly reading

2015-08-12T00:49:18Z

Susheng：

*[[Speech Group Reading|张之勇 2014-12-28 APSIPA paper reading]]

*[http://www.jmlr.org/proceedings/papers/v32/graves14.pdf 刘超2015-03-11 Towards End-to-End Speech Recognition with Recurrent Neural Networks]

*[[媒体文件:2015 Building DNN Acoustic Models for Large Vocabulary Speech Recognition.pdf|汤志远2015-3-18 - Building DNN Acoustic Models for Large Vocabulary Speech Recognition]]

*[[媒体文件:CONTRASTIVE AUTO-ENCODER FOR PHONEME RECOGNITION.pdf|林一叶2015-4-1 - CONTRASTIVE AUTO-ENCODER FOR PHONEME RECOGNITION]]

*[[媒体文件:2014 speech dereverberation using weighted prediction error with laplacian model of the designed signal.pdf|张雪薇2015-4-1 - speech dereverberation using weighted prediction error with laplacian model of the designed signal]]

*[http://arxiv.org/pdf/1312.6184v7.pdf 王东2015-4-1 - Do Deep Nets Really Need to be Deep?]

*[http://arxiv.org/pdf/1503.02531v1.pdf 王东2015-4-1 - Distilling the Knowledge in a Neural Network]

*[[媒体文件:Neural_Network_Acoustic_Models_with_Superviesed_Hidden_Layers_for_Automatic_Speech_Recognition.pdf|殷实2015-4-8 - Neural_Network_Acoustic_Models_with_Superviesed_Hidden_Layers_for_Automatic_Speech_Recognition]]

*[[媒体文件:Ensemble_Deep_Learning_for_Speech_Recognition.pdf|赵梦原2015-4-8 - Ensemble_Deep_Learning_for_Speech_Recognition]]

*[[媒体文件:An evaluation of target speech for a nonaudible murmur enhancement system in noisy enviroment.pdf|曾翔宇2015-3-11 - An evaluation of target speech for a nonaudible murmur enhancement system in noisy enviroment]]

*[http://www.kecl.ntt.co.jp/icl/signal/hori/publications/thori_icassp2014.pdf 刘超2015-04-14 Real-time one-pass decoding with recurrent neural network language model for speech recognition]

*[[媒体文件:2014 Reshaping deep neural network for fast decoding by node-pruning.pdf|汤志远 2015-04-29 Reshaping deep neural network for fast decoding by node-pruning]]

*[[媒体文件:06843244.pdf|张雪薇2015-4-29 SPEECH DEREVERBERATION WITH MULTI-CHANNEL LINEAR PREDICTION AND SPARSE PRIORS FOR THE DESIRED SIGNAL]]

*[[媒体文件:0000096.pdf|张雪薇2015-4-29 MULTI-CHANNEL LINEAR PREDICTION-BASED SPEECH DEREVERBERATION WITH LOW-RANK POWER SPECTROGRAM APPROXIMATION]]

*[[媒体文件:0000096.pdf|张之勇2015-5-6 On the importance of initialization and momentum in deep learning.pdf]]

*[[媒体文件:ICASSP.rar|张雪薇2015-5-20 ICASSP_dereveration]]

*[[媒体文件:ICASSP Selected Readings.rar|汤志远2015-5-27 ICASSP selected readings]]

*[[媒体文件:2015_Submodular data selection with acoustic and phonetic features for automatic speech recognition.pdf|张之勇2015-6-24 2015_Submodular data selection with acoustic and phonetic features for automatic speech recognition.pdf]]

*[[媒体文件:SPEECH DEREVERBERATION USING A LEARNED SPEECH MODEL.pdf|张雪薇2015-7-1 SPEECH DEREVERBERATION USING A LEARNED SPEECH MODEL.pdf]]

*[http://arxiv.org/abs/1504.01482 汤志远2015-7-8 Deep Recurrent Neural Networks for Acoustic Modelling]

*[http://arxiv.org/abs/1503.04069 汤志远2015-7-8 LSTM: A Search Space Odyssey]

*[http://arxiv.org/abs/1507.01526 汤志远2015-7-8 Grid Long Short-Term Memory]

*[http://arxiv.org/abs/1506.02078 汤志远2015-7-8 Visualizing and Understanding Recurrent Networks]

*[[媒体文件:annealed_dropout_trained_maxout_networks_for_improved_lvcsr.pdf|赵梦原2015-7-8 Annealed Dropout_trained Maxout Networks for Improved LVCSR]]

*[[媒体文件:automatic_pronunciation_verification_for_speech_recognition.pdf|赵梦原2015-7-8 Automatic Pronunciation Verification for Speech Recognition]]

*[[媒体文件:2015_Frame-by-frame_language_identification_in_short_utterances_using_deep_neural_networks.pdf|张雪薇2015-7-16 2015_Frame-by-frame_language_identification_in_short_utterances_using_deep_neural_networks]]

*[[媒体文件:SEQUENCE_CLASSIFICATION_USING_THE_HIGH-LEVEL_FEATURES_EXTRACTED_.pdf|张雪薇2015-7-16 SEQUENCE_CLASSIFICATION_USING_THE_HIGH-LEVEL_FEATURES_EXTRACTED]]

*[http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf 王东2015-7-22 What is Wrong with Deep Learning (Yann Lecun at CVPR 2015)]

*[http://videolectures.net/icml09_bengio_lecun_tldar/ 王东2015-7-22 A tutorial from Bengio and Lecun]

*[[asr-read-icml|王东2015-7-22 ICML 2015 reading list]]

*[http://jmlr.org/proceedings/papers/v37/ioffe15.pdf 王东2015-7-22 Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift]

*[http://cslt.riit.tsinghua.edu.cn/cgi-bin/cvss/cvss_request.pl 王东2015-7-24 No initial learning rate anymore]

*[http://jmlr.org/proceedings/papers/v37/long15.pdf 王东2015-7-29 Learning Transferable Features with Deep Adaptation Networks]

*[https://personalrobotics.ri.cmu.edu/courses/papers/Amari1998a.pdf 王东2015-7-29 Natural Gradient Works Efficiently in Learning]

*[http://arxiv.org/abs/1206.5533 王东2015-7-29 Practical Recommendations for Gradient-Based Training of Deep Architectures]

*[http://link.springer.com/chapter/10.1007%2F978-3-642-35289-8_3 王东2015-7-29 Efficient Backprop]
*[[媒体文件:2015_Batch normalization Accelerating deep network training by reducing internal covariate shift.pdf|张之勇2015-7-31 2015_Batch normalization Accelerating deep network training by reducing internal covariate shift.pdf]]

*[[媒体文件:A_time_delay_neural_network_architecture_for_efficient_modeling_of_long_temporal_contexts.pdf|赵梦原2015-7-29 A time delay neural network architecture for efficient modeling of long temporal contexts.pdf]]

*[[媒体文件:1-s2.0-S0885230814000114-main.pdf|张雪薇2015-8-05 Feature enhancement by deep LSTM networks for ASRin reverberant multisource environments]]

*[[媒体文件:Recurrent Neural Networks for Noise Reduction in Robust ASR.pdf|张雪薇2015-8-05 Recurrent Neural Networks for Noise Reduction in Robust ASR]]

*[[媒体文件:2015 Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition.pdf|汤志远 2015-8-05 Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition]]

*[[媒体文件:CUDA_C_Programming_Guide.pdf|苏圣 2015-8-012 CUDA_C_Programming_Guide.pdf]]

*[[媒体文件:CUBLAS_Library.pdf|苏圣 2015-8-012 CUBLAS_Library.pdf]]

文件:CUBLAS Library.pdf

2015-08-12T00:44:55Z

Susheng：CUBLAS_Library.pdf

CUBLAS_Library.pdf

文件:CUDA C Programming Guide.pdf

2015-08-12T00:44:29Z

Susheng：CUDA_C_Programming_Guide.pdf

CUDA_C_Programming_Guide.pdf

Sheng Su 15-08-11

2015-08-11T02:54:09Z

Susheng：以“Last week : 1.Debug multi-GPU program 2.Learning CUDA This week : 1.Debug multi-GPU program 2.Learning cuBlas”为内容创建页面

Last week : 1.Debug multi-GPU program 2.Learning CUDA

This week : 1.Debug multi-GPU program 2.Learning cuBlas

2015-08-10

2015-08-11T02:51:21Z

Susheng：

[[Mengyuan Zhao 15-08-10]]

[[Xiaoxi Wang 15-08-10]]

[[Xuewei Zhang 15-08-10]]

[[Zhiyuan Tang 15-08-10]]

[[Chao Xing 15-08-10]]

[[Tianyi Luo 15-08-10]]

[[Haichao Yu 15-08-10]]

[[Sheng Su 15-08-11]]

Sheng su 15-08-05

2015-08-05T11:27:29Z

Susheng：以“Weekly Report: Write code, use multi-GPU to train DNN. Target of next week: Continue writing code, and debug.”为内容创建页面

Weekly Report:

Write code, use multi-GPU to train DNN.

Target of next week:

Continue writing code, and debug.

2015-08-02

2015-08-05T11:22:53Z

Susheng：

[[Zhiyong Zhang 08-02]]

[[Mengyuan Zhao 15-08-03]]

[[Xuewei Zhang 15-08-03]]

[[Rong Liu 15-08-03]]

[[Tianyi Luo 15-08-03]]

[[Hui Tang 15-08-03]]

[[Yang Yu 15-08-03]]

[[Sheng su 15-08-05]]