Novel method for feature selection based on entropy

doi:10.3778/j.issn.1002-8331.2009.15.016

Computer Engineering and Applications ›› 2009, Vol. 45 ›› Issue (15): 54-57.DOI: 10.3778/j.issn.1002-8331.2009.15.016

• 研究、探讨 • Previous Articles Next Articles

Novel method for feature selection based on entropy

LI Yang-huan,GAO Feng,LI Teng,ZHOU Zhi-min

School of Electronic Science and Engineering，NUDT，Changsha 410073，China

Received:2008-03-31 Revised:2008-06-30 Online:2009-05-21 Published:2009-05-21
Contact: LI Yang-huan

特征选择中信息熵的应用

李杨寰,高峰,李腾,周智敏

国防科技大学电子科学与工程学院，长沙 410073

通讯作者: 李杨寰

Abstract

Abstract: This paper applies the entropy concept of information theory to feature selection，and defines two information measurements overlap entropy and error entropy that are used to evaluate features.Then the physical meanings of the two definiens are explained.The crucial problem about how to divide the interzones is provided in the following.Then this paper adds a kind of similarity to form a complete feature selection process in order to involve the defect of entropy that it can not filter similar features.An emulation is made to prove the validity of this method at the end.

Key words: feature selection, overlap-entropy, erro-entropy, similarity, feature evalution, information measurement

摘要： 将信息论中熵的概念应用到特征选择中，定义了两种信息测度评价特征——误差熵和混叠熵，然后阐述了两种定义的不用物理意义，分析了计算熵中最关键的区间划分问题，并提出一种较好的区间划分方法。由于熵不能将相似的特征进行剔除，结合相似系数提出了一套完整的基于熵的特征选择过程，并通过仿真实验进行验证。

关键词: 特征选择, 混叠熵, 误差熵, 相似度, 特征评价, 信息测度

LI Yang-huan,GAO Feng,LI Teng,ZHOU Zhi-min. Novel method for feature selection based on entropy[J]. Computer Engineering and Applications, 2009, 45(15): 54-57.

李杨寰,高峰,李腾,周智敏. 特征选择中信息熵的应用[J]. 计算机工程与应用, 2009, 45(15): 54-57.

[1]	ZHANG Qishan, CHEN Lulu. Slope One Algorithm Based on Grey Correlational Analysis by Method of Degree of Balance and Approach [J]. Computer Engineering and Applications, 2021, 57(9): 96-102.
[2]	WANG Yonggui, LI Qianyu. Hybrid Collaborative Filtering Recommendation Algorithm Based on KNN-GBDT [J]. Computer Engineering and Applications, 2021, 57(9): 103-108.
[3]	ZHANG Songcan, PU Jiexin, SI Yanna, SUN Lifan. Adaptive Improved Ant Colony Algorithm Based on Population Similarity and Its Application [J]. Computer Engineering and Applications, 2021, 57(8): 70-77.
[4]	ZHANG Xiaowen, REN Yongfeng. Image Matching Algorithm Combining Sparse Representation and Topological Similarity [J]. Computer Engineering and Applications, 2021, 57(8): 198-203.
[5]	LI Li, JI Xinyuan, SONG Song. Prediction Model for Number of Software Defects in Loop [J]. Computer Engineering and Applications, 2021, 57(7): 158-163.
[6]	LI Jingxing, YANG Youlong. Feature Selection of Markov Blanket for High Dimensional Data [J]. Computer Engineering and Applications, 2021, 57(6): 58-66.
[7]	YANG Fang, YIN Xi, SI Jianhui, LIU Hongyuan, WANG Xue. Mathematical Expression Similarity Calculation Method Based on Focus Clustering [J]. Computer Engineering and Applications, 2021, 57(6): 88-93.
[8]	QIAN Yunyun, YANG Wenzhong, YAO Miao, LI Hailei, CHAI Yachuang. Topic Community Discovery Model Incorporating Topic Similarity Weight [J]. Computer Engineering and Applications, 2021, 57(5): 107-114.
[9]	JIANG Bin, LIANG Xiao’an, ZHANG Liang, GAO Yangjun. Evidence Combination Method Based on Improved Modified Weight [J]. Computer Engineering and Applications, 2021, 57(24): 100-106.
[10]	TIAN Wei’an, CHEN Hongmei, ZHOU Lihua. Diversified Recommendation Method Based on Similar Users’Curiosity [J]. Computer Engineering and Applications, 2021, 57(23): 113-121.
[11]	LIN Weixing, WANG Yujia, CHEN Wanfen, LIANG Haina. High-Dimensional Data Feature Selection Algorithm Based on Multifactor Particle Swarm Optimization [J]. Computer Engineering and Applications, 2021, 57(22): 199-207.
[12]	LI Longzhu, LIN Yaojin, LYU Yan, LU Shun, WANG Chenxi. Online Streaming Feature Selection Algorithm Using Neighborhood Information Interaction [J]. Computer Engineering and Applications, 2021, 57(21): 102-108.
[13]	CHEN Qianru, LI Yali, XU Kequan, LIU Yilong, WANG Shuqin. WKNN Feature Selection Method Based on Self-Tuning Adaptive Genetic Algorithm [J]. Computer Engineering and Applications, 2021, 57(20): 164-171.
[14]	LIANG Tian, CAO Dexin. Improved and Simplified Particle Swarm Optimization Algorithm Based on Levy Flight [J]. Computer Engineering and Applications, 2021, 57(20): 188-196.
[15]	WEI Dingfeng, LI Liang, CHAI Jing. Social Recommendation Algorithm by Fusing Item Information [J]. Computer Engineering and Applications, 2021, 57(19): 198-204.

Novel method for feature selection based on entropy

特征选择中信息熵的应用

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics