计算机工程与应用 ›› 2014, Vol. 50 ›› Issue (6): 127-131.

• 数据库、数据挖掘、机器学习 • 上一篇    下一篇

一种中文领域概念词自动提取方法研究

董丽丽1,李  欢1,张  翔1,刘闫锋2   

  1. 1.西安建筑科技大学 信息与控制工程学院,西安 710055
    2.陕西学前师范学院,西安 710100
  • 出版日期:2014-03-15 发布日期:2015-05-12

Method for automatic extraction of Chinese domain concepts

DONG Lili1, LI Huan1, ZHANG Xiang1, LIU Yanfeng2   

  1. 1.College of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an 710055, China
    2.Shaanxi Xueqian Normal University, Xi’an 710100, China
  • Online:2014-03-15 Published:2015-05-12

摘要: 针对统计学方法在领域概念获取时缺少词语语义信息的问题,提出了一种结合语义相似度和改进近邻传播算法的领域概念自动获取方法。该方法通过互信息进行合成词提取,使用对数似然比避免对低频词的遗漏,利用HowNet和余弦相似度识别术语间同义词,采用改进的近邻传播算法获取领域概念集合。实验结果表明,该方法在准确率、召回率和困惑度变化率上比传统的方法都有较大提高。

关键词: 领域概念获取, 改进近邻传播算法, 对数似然比, 语义相似度, 互信息

Abstract: For statistical method lacks semantic information between words in domain concepts extraction, this paper presents a domain concept automatic extraction method, which combines semantic similarity and improved affinity propagation. The compound words are extracted by using mutual information, and then the log-likelihood is used to avoid the omission of low-frequency words, after that the synonyms between terms are identified by using HowNet and the cosine similarity. The improved affinity propagation algorithm is used to obtain the collection of domain concepts. The experimental results show that the method has higher accuracy, recall rate, and perplexity change ratio than the traditional method.

Key words: domain concept extraction, improved affinity propagation, log-likelihood, semantic similarity, mutual information