计算机工程与应用 ›› 2010, Vol. 46 ›› Issue (36): 147-149.DOI: 10.3778/j.issn.1002-8331.2010.36.040

• 数据库、信号与信息处理 • 上一篇    下一篇

改进的HMM应用于哈萨克语词性标注

侯呈风,古丽拉·阿东别克   

  1. 新疆大学 信息科学与工程学院,乌鲁木齐 830046
  • 收稿日期:2010-07-21 修回日期:2010-09-10 出版日期:2010-12-21 发布日期:2010-12-21
  • 通讯作者: 侯呈风

Improved hidden Markov models used in Kazakh part-of-speech tagging

HOU Cheng-feng,Gulila·Altenbek   

  1. College of Information Science and Engineering,Xinjiang University,Urumqi 830046,China
  • Received:2010-07-21 Revised:2010-09-10 Online:2010-12-21 Published:2010-12-21
  • Contact: HOU Cheng-feng

摘要: 哈萨克语的词性标注在自然语言信息处理领域中扮演着重要角色,是句法分析、信息抽取、机器翻译等自然语言处理的基础。在传统的HMM的基础上改进了HMM模型参数的计算、数据平滑以及未登录词的处理方法,使之更好地体现词语的上下文依赖关系。利用基于统计的方法对哈萨克语熟语料进行训练,然后用Viterbi算法实现词性标注。实验结果表明利用改进的HMM进行词性标注的效果比传统的HMM好。

关键词: 隐马尔科夫模型, 哈萨克语, 词性标注

Abstract: Part-of-Speech(POS) tagging of Kazakh is playing a key role in natural language information processing.Kazakh POS tagging is the basis of syntactic analysis,information retrieval and machine translation.Based upon the traditional HMM,computing of HMM parameters,data-smoothing and process of?words which are not logged enable to improve context dependence relationship.Use statistical method to train Kazakh corpus,and then use Viterbi algorithm to implement POS tagging.The experimental results show that the effect of POS tagging of improved HMM is better than traditional HMM.

Key words: Hidden Markov Models(HMM), Kazakh, part-of-speech tagging

中图分类号: