计算机工程与应用 ›› 2017, Vol. 53 ›› Issue (23): 34-38.DOI: 10.3778/j.issn.1002-8331.1607-0312

• 理论与研发 • 上一篇    下一篇

改进信息增益的维吾尔文特征选择方法

韩军兵,哈力旦·阿布都热依木,古力努尔·艾尔肯,何  燕   

  1. 新疆大学 电气工程学院,乌鲁木齐 830047
  • 出版日期:2017-12-01 发布日期:2017-12-14

Improved information gain algorithm based on Uyghur feature selection

HAN Junbing, Halidan·Abudureyimu, Gulnur·Arken, HE Yan   

  1. College of Electrical Engineering, Xinjiang University, Urumqi 830047, China
  • Online:2017-12-01 Published:2017-12-14

摘要: 特征选择是维吾尔语文本分类的关键技术,对分类结果将产生直接的影响。为了提高传统信息增益在维吾尔文特征选择中的效果,在深度分析维吾尔文语种特点的基础上,提出了一种新的信息增益特征选择方法。该方法结合类词频和特征分布系数以及倒逆文档频率,对传统信息增益进行修正;引入一个备选特征分布系数来平衡类间选取的特征个数;在维吾尔文数据集上实验验证。实验结果表明,改进的算法对维吾尔文分类效果有明显的提高。

关键词: 文本分类, 信息增益, 类词频, 倒逆转文档频率, 特征选择

Abstract: Feature selection is the key step of Uyghur text classification, which causes direct effect on the categorization results. To improve the effect of traditional information gain algorithm on the Uyghur feature selection, a new information gain feature selection method is proposed on the basis of deep?analysis of Uyghur text feature. This method combines with word frequency in class, characteristics of the distribution coefficient and inverse document frequency, thus traditional information gain is modified. Furthermore, it introduces an alternative features of distribution coefficient to balance the selected number between the classes. Finally, ?experimental verification?is conducted on Uyghur text dataset. The results show that modified information gain algorithm has greatly improved the effect of Uyghur text classification.

Key words: text classification, information gain, word frequency in class, inverse document frequency, feature selection