Computer Engineering and Applications ›› 2013, Vol. 49 ›› Issue (22): 110-114.

Previous Articles     Next Articles

Improved CHI text feature selection based on word frequency information

LIU Haifeng, SU Zhan, LIU Shousheng   

  1. Institute of Sciences, PLA University of Science and Technology, Nanjing 210007, China
  • Online:2013-11-15 Published:2013-11-15

一种基于词频信息的改进CHI文本特征选择

刘海峰,苏  展,刘守生   

  1. 解放军理工大学 理学院,南京 210007

Abstract: CHI is a commonly used text feature selection method. Aiming at the shortcomings of the model, according to the frequency characteristic, the CHI model is gradually optimized from the feature distribution within class, distribution between class and the distribution between different text in the same category. This approach makes the characteristic frequency information has been used effectively. An improved CHI model based on word frequency information is proposed. The text categorization experiment subsequently proves the validity of the new optimized CHI model.

Key words: text categorization, feature selection, Chi-square, distribution within class, distribution between class

摘要: CHI是一种常用的文本特征选择方法。针对该模型的不足之处,以特征项的频数为依据,分别从特征项的类内分布、类间分布以及类内不同文本之间分布等角度,对CHI模型进行逐步优化,使得特征项频数信息得到了有效利用。提出了一种基于词频信息的改进CHI模型。随后的文本分类试验证明了提出优化CHI模型的有效性。

关键词: 文本分类, 特征选择, [&chi, 2]统计, 类内分布, 类间分布