Computer Engineering and Applications ›› 2014, Vol. 50 ›› Issue (21): 205-209.

Previous Articles     Next Articles

Rank aggregation-based Kazakh fixed phrases extraction

SANG Haiyan1,2, Gulia·ALTENBEK1,2, SUN Ruina3, CHEN Li1,2   

  1. 1.College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
    2.The Base of Kazakh and Kirghiz Language of National Language Resource Monitoring and Research Center Minority Languages, Urumqi 830046, China
    3.College of Statistical Information, Xinjiang University of Finance and Economics, Urumqi 830046, China
  • Online:2014-11-01 Published:2014-10-28

基于排序集成的哈萨克语固定短语抽取

桑海岩1,2,古丽拉·阿东别克1,2,孙瑞娜3,陈  莉1,2   

  1. 1.新疆大学 信息科学与工程学院,乌鲁木齐 830046
    2.国家语言资源监测与研究中心少数民族语言中心 哈萨克和柯尔克孜语文基地,乌鲁木齐 830046
    3.新疆财经大学 统计信息学院,乌鲁木齐 830046

Abstract: Phrase extraction plays a key role in text information understanding, such as automatic text classification, topic extraction, and analysis of patent search, etc. As the part of phrase research, the fixed phrase extraction has important practical significance on natural language processing tasks including the lexicographer. The Kazakh is agglutinative language, rich in inflections. These characteristics of the Kazakh bring certain difficulties to fixed phrase extraction. This paper proposes a general fixed phrase extraction algorithm. The algorithm considers the fixed phrase extraction as a scheduling problem, uses C-value, mutual information and log-likelihood statistics to extract and schedule, and presents a new rank aggregation method to obtain a scheduling result set. The experimental results indicate that the algorithm gets higher accuracy compared with popular signal extraction algorithms.

Key words: natural language processing, fixed phrases, rank aggregation, mutual information, log-likelihood, C-value

摘要: 短语抽取是文本自动分类、主题提取及专利检索分析等文本信息理解等工作中都要应用到的一项关键技术。固定短语抽取作为短语研究的一部分,对短语标注、辞典编撰等自然语言处理任务都具有重要的现实意义。哈萨克语是黏着语,词形变化丰富,这些特点给哈语固定短语的抽取带来了一定的困难。提出一个总体的固定短语抽取算法,把固定短语抽取看作一个排序问题,使用C-value、互信息和log-likelihood进行抽取排序,并设计了一个新的排序集成方法对抽取的结果进行集成。实验分析结果表明,与单独的抽取算法比较,该算法达到了更高的准确率。

关键词: 自然语言处理, 固定短语, 排序集成, 互信息, 似然比, C-value算法