“Hulan-2014-10-31”版本间的差异

来自cslt Wiki
跳转至: 导航搜索
improve lucene search
Lr讨论 | 贡献
第29行: 第29行:
 
:* check the query method for getting lucene information and to rewrite the score method like the idf value.  
 
:* check the query method for getting lucene information and to rewrite the score method like the idf value.  
  
*IDF  
+
*IDF(caoli)
 
:* test the different idf vale from baidu sougou in fuzzymatch.
 
:* test the different idf vale from baidu sougou in fuzzymatch.
 
+
:* IDF from train-data performance bad than default IDF,from 0.63->0.69.
 +
==knowledge structure==
 +
* structure the default answer using attributes of the entity.
 
==plan to discuss==
 
==plan to discuss==

2014年10月31日 (五) 04:59的版本

Dialog system

Algorithm

Spell mistake

  • using ngram to get candidate sentence.

improve lucene search

  • lucene similarity method
different result in lucene
method Default BM25 LMDirichlet DFR LMJelinekMercer IB
Accary 0.66228 0.66228 0.4091 0.65476 0.65476 0.6666
  • our vsm method
  • our vsm method re-rank(54%),lucene(67%)
  • lucene top50(caoli)
  • top10(82.95%),top20(86.34),top50(90.22%)
  • need to check the other 10% error
  • lucene Optimization(liurong)
  • rewrite the method to select the 50 standard question not same template.
  • test the boost keyword weight and extract the synonyms word.
  • check the word segment for template.
  • min-segment method improve the accuracy.
  • check the query method for getting lucene information and to rewrite the score method like the idf value.
  • IDF(caoli)
  • test the different idf vale from baidu sougou in fuzzymatch.
  • IDF from train-data performance bad than default IDF,from 0.63->0.69.

knowledge structure

  • structure the default answer using attributes of the entity.

plan to discuss