Online Streaming Feature Selection Algorithm Using Neighborhood Information Interaction

doi:10.3778/j.issn.1002-8331.2010-0096

Abstract

Abstract:

In the open dynamic environment, the task of machine learning faces the high dimensionality and dynamicity of feature space. At present, the existing online streaming feature selection algorithms generally consider the importance of feature and the redundancy between features, and ignore the interaction between features. Feature interaction denotes a feature irrelevant or weakly relevant with the labels by itself, but when it is combined with some other features, it will be strongly correlated with the labels. Based on this, an online streaming feature selection algorithm based on neighborhood information interaction is proposed, which includes online interaction feature selection and online redundant feature deletion, i.e.， calculating the interaction strength between the new arrived feature and the selected feature subset, and deleting redundant features using pair-wise comparison mechanism. Finally, extensive experiments are conducted on ten data sets, and the results show the proposed algorithm is effective.

Key words: feature selection, streaming feature, feature interaction, neighborhood mutual information

摘要：

开放动态环境下的机器学习任务面临着数据特征空间的高维性和动态性。目前已有在线流特征选择算法基本仅考虑特征的重要性和冗余性，忽略了特征的交互性。特征交互是指那些本身与标签单独统计时呈现无关或弱相关，但与其他特征结合时却能与标签呈强相关的特征。基于此，提出一种基于邻域信息交互的在线流特征选择算法，该算法分为在线交互特征选择和在线冗余特征剔除两个阶段，即直接计算新到特征与整个已选特征子集的交互强弱程度，以及利用成对比较机制剔除冗余特征。在10个数据集上的实验结果表明了所提算法的有效性。

关键词: 特征选择, 流特征, 特征交互, 邻域互信息

LI Longzhu, LIN Yaojin, LYU Yan, LU Shun, WANG Chenxi. Online Streaming Feature Selection Algorithm Using Neighborhood Information Interaction[J]. Computer Engineering and Applications, 2021, 57(21): 102-108.

李珑珠，林耀进，吕彦，卢舜，王晨曦. 利用邻域信息交互的在线流特征选择算法[J]. 计算机工程与应用, 2021, 57(21): 102-108.

References

[1] PUN C，YAN C，YUAN X.Robust image hashing using progressive feature selection for tampering detection[J].Multimedia Tools and Applications，2018，77（10）：1-25.
[2] ISABELLE B，CASSIE B，EKATERINA C，et al.Feature selection and machine learning based multilevel stress detection from ECG signals[C]//International Conference on Innovation in Medicine and Healthcare，2018.
[3] JORNSTEN R，YU B.Simultaneous gene clustering and subset selection for sample classification via MDL[J].Bioinformatics，2003，19（9）：1100-1109.
[4] WU X，YU K，DING W，et al.Online feature selection with streaming features[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2013，35（5）：1178-1192.
[5] DING W，STEPINSKI T，MU Y，et al，Sub-kilometer crater discovery with boosting and transfer learning[J].ACM Transactions on Intelligent Systems and Technology，2011，2：1-22.
[6] ZHOU J，FOSTER D P，STINE R A，et al.Streamwise feature selection[J].Journal of Machine Learning Research，2006，7（1）：1861-1885.
[7] WU X，YU K，WANG H，et al.Online streaming feature selection[C]//International Conference on Machine Learning，2010：1159-1166.
[8] YU K，WU X D，DING W，et al.Scalable and accurate online feature selection for big data[J].ACM Transactions on Knowledge Discovery from Data，2016，11（2）：1-39.
[9] ZHOU P，HU X，LI P，et al.Online feature selection for high-dimensional class-imbalanced data[J]Knowledge-Based Systems，2017，136（15）：187-199.
[10] ZHOU P，HU X，LI P.A new online feature selection method using neighborhood rough set[C]//IEEE International Conference on Big Knowledge，2017：135-142.
[11] 陈祥焰，林耀进，王晨曦.基于邻域粗糙集的高维类不平衡数据在线流特征选择[J].模式识别与人工智能，2019，32（8）：726-735.
CHEN X Y，LIN Y J，WANG C X.Online streaming feature selection for high-dimensional and class-imbalanced data based on neighborhood rough set[J].Pattern Recognition and Artificial Intelligence，2019，32（8）：726-735.
[12] 白盛兴，林耀进，王晨曦，等.基于邻域粗糙集的大规模层次分类在线流特征选择[J].模式识别与人工智能，2019，32（9）：811-820.
BAI S X，LIN Y J，WANG C X，et al.Large-scale hierarchical classification online streaming feature selection based on neighborhood rough set[J].Pattern Recognition and Artificial Intelligence，2019，32（9）：811-820.
[13] LIN Y，HU Q，ZHANG J，et al.Multi-label feature selection with streaming labels[J].Information Sciences，2016，372：256-275.
[14] LIN Y，HU Q，LIU J，et al.Streaming feature selection for multi-label learning based on fuzzy mutual information[J].IEEE Transactions on Fuzzy Systems，2017，25（6）：1491-1507.
[15] LIU J，LIN Y，LI Y，et al.Online multi-label streaming feature selection based on neighborhood rough set[J].Pattern Recognition，2018，84：273-287.
[16] LIU J，LI Y，WENG W，et al.Feature selection for multi-label learning with streaming label[J].Neurocomputing，2020，387：268-278.
[17] NAKARIYAKUL S.High-dimensional hybrid feature selection using interaction information-guided search[J].Knowledge-Based Systems，2018，145：59-66.
[18] HU Q，ZHANG L，ZHANG D，et al.Measuring relevance between discrete and continuous features based on neighborhood mutual information[J].Expert Systems with Applications，2011，38：10737-10750.
[19] KANNAN S，RAMARAJ N.A novel hybrid feature selection via symmetrical uncertainty ranking based local memetic search algorithm[J].Knowledge-Based Systems，2010，23（6）：580-585.
[20] WANG C X，LIN Y J，LIU J H.Feature selection for multi-label learning with missing labels[J].Applied Intelligence，2019，49：3027-3042.
[21] 王晨曦，林耀进，刘景华，等.基于最近邻互信息的特征选择算法[J].计算机工程与应用，2016，52（18）：74-78.
WANG C X，LIN Y J，LIU J H，et al.Feature selection algorithm based on nearest-neighbor mutual information[J].Computer Engineering and Applications，2016，52（18）：74-78.