“ASR:2014-12-29”版本间的差异

来自cslt Wiki
跳转至: 导航搜索
Domain specific LM
Lr讨论 | 贡献
improve lucene search
第54行: 第54行:
 
====improve lucene search====
 
====improve lucene search====
 
:* add more feature to improve search.
 
:* add more feature to improve search.
::*  
+
::* POS, NER ,tf ,idf ..
 +
 
 
====XiaoI framework====
 
====XiaoI framework====
 
*  context in xiaoI
 
*  context in xiaoI

2014年12月29日 (一) 06:23的版本

Text Processing

LM development

Domain specific LM

  • LM2.0
  • data check for lexicon(jietong)
  • merge lm with NAME POI etc.(hanzhenglong/wxx)
  • mix the sougou2T-lm,kn-discount continue
  • train a large lm using 25w-dict.(hanzhenglong/wxx)
  • prun history lm(wxx)
  • new dict.
  • dongxu help zhenglong with large dictionary.

tag LM

  • need to do
  • tag Probability should test add the weight(hanzhenglong) and handover to hanzhenglong (hold)
paper
  • modify the paper(yuanb two days),paper submit this week.

RNN LM

  • rnn
  • test wer RNNLM on Chinese data from jietong-data(this week)
  • generate the ngram model from rnnlm and test the ppl with different size txt.[1]
  • lstm+rnn
  • check the lstm-rnnlm code about how to Initialize and update learning rate.(hold)

Word2Vector

W2V based doc classification

  • Initial results variable Bayesian GMM obtained. Performance is not as good as the conventional GMM.(hold)
  • Non-linear inter-language transform: English-Spanish-Czch: wv model training done, transform model on investigation

Knowledge vector

  • Knowledge vector
  • Make a proper test set.
  • Modify the object function and training process.
  • Read Liu's paper.

relation

  • Accomplish transE with almost the same performance as the paper did(even better)[2]

Character to word

  • Character to word conversion(hold)
  • prepare the task: word similarity
  • prepare the dict.

Translation

  • v5.0 demo released
  • cut the dict and use new segment-tool

QA

improve fuzzy match

  • add Synonyms similarity using MERT-4 method(hold)

improve lucene search

  • add more feature to improve search.
  • POS, NER ,tf ,idf ..

XiaoI framework

  • context in xiaoI

query normalization

  • using NER to normalize the word
  • new inter will install SEMPRE