KNN特征增强与互信息特征选择的两阶段多维分类方法

doi:10.3778/j.issn.1002-8331.2412-0388

摘要/Abstract

摘要： 现有多维分类的特征增强方法虽丰富了特征空间，但对特征内在质量缺乏有效评估，易引入冗余，影响分类性能。提出基于KNN特征增强与互信息特征选择的两阶段多维分类方法KMFM。第一阶段通过KNN特征增强扩展特征空间，第二阶段基于互信息评估并筛选相关性最强的特征子集，且通过计算类别空间组合熵考虑类别变量间的依赖关系。在10个基准数据集上的实验结果表明，KMFM在汉明分值、精确匹配和亚精确匹配指标上相比现有方法取得显著提升。在90种配置中，KMFM实现77.8%的最佳表现；与只采用特征增强的KRAM相比，性能提升显著；与只进行互信息特征选择MIFS相比，分类性能在9个指标上全面优越，充分说明了该算法的有效性和泛用性。

关键词: 多维分类, 特征增强, 特征选择, 互信息, 类依赖

Abstract: Although the existing feature enhancement methods for multi-dimensional classification enrich the feature space, they lack effective evaluation of the inherent quality of features and are prone to introducing redundancy, which affects classification performance. Therefore, a two-stage multi-dimensional classification method KMFM is proposed. In the first stage, the feature space is expanded through KNN feature enhancement. In the second stage, the feature subset with the strongest correlation is evaluated and selected based on mutual information, and the dependency relationship between category variables is considered by calculating the combination entropy of the category space. The experimental results on ten benchmark datasets show that KMFM achieves significant improvements in Hamming score, exact match, and sub-exact match metrics compared with existing methods. Among the 90 configurations, KMFM achieves the best performance in 77.8%. Compared with KRAM, which only uses feature enhancement, the performance of KMFM is significantly improved. Compared with MIFS, which only performs mutual information feature selection, the classification performance of KMFM is comprehensively superior in nine metrics, fully demonstrating the effectiveness and universality of this algorithm.

Key words: multi-dimensional classification, feature augmentation, feature selection, mutual information, class dependency

李二超, 张宝新, 贾彬彬. KNN特征增强与互信息特征选择的两阶段多维分类方法[J]. 计算机工程与应用, 2025, 61(15): 167-177.

LI Erchao, ZHANG Baoxin, JIA Binbin. Two-Stage Multi-Dimensional Classification Method Combining KNN Feature Enhancement and Mutual Information Feature Selection[J]. Computer Engineering and Applications, 2025, 61(15): 167-177.

参考文献

[1] GRANDINI M, BAGLI E, VISANI G. Metrics for multi-class classification: an overview[J]. arXiv:2008.05756, 2020.
[2] SHATKAY H, PAN F X, RZHETSKY A, et al. Multi-dimensional classification of biomedical text: toward automated, practical provision of high-utility text to diverse users[J]. Bioinformatics, 2008, 24(18): 2086-2093.
[3] RODRIGUEZ J D, PEREZ A, ARTETA D, et al. Using multidimensional Bayesian network classifiers to assist the treatment of multiple sclerosis[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C(Applications and Reviews), 2012, 42(6): 1705-1715.
[4] AL MUKTADIR A H, MIYAZAWA T, MARTINEZ-JULIA P, et al. Multi-target classification based automatic virtual resource allocation scheme[J]. IEICE Transactions on Information and Systems, 2019, 102(5): 898-909.
[5] VERMA S P, USCANGA-JUNCO O A, DíAZ-GONZáLEZ L. A statistically coherent robust multidimensional classification scheme for water[J]. Science of the Total Environment, 2021, 750: 141704.
[6] HUANG T, JIA B B, ZHANG M L. Deep multi-dimensional classification with pairwise dimension-specific features[C]//Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 2024: 4183-4191.
[7] JIA B B, ZHANG M L. Multi-dimensional classification via stacked dependency exploitation[J]. Science China Information Sciences, 2020, 63(12): 222102.
[8] ZHANG M L, LI Y K, LIU X Y, et al. Binary relevance for multi-label learning: an overview[J]. Frontiers of Computer Science, 2018, 12(2): 191-202.
[9] READ J, BIELZA C, LARRA?AGA P. Multi-dimensional classification with super-classes[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1720-1733.
[10] TSOUMAKAS G, KATAKIS I, VLAHAVAS I. Random [k]-labelsets for multilabel classification[J]. IEEE Transactions on Knowledge and Data Engineering, 2011, 23(7): 1079-1089.
[11] GIL-BEGUE S, BIELZA C, LARRA?AGA P. Multi-dimensional Bayesian network classifiers: a survey[J]. Artificial Intelligence Review, 2021, 54(1): 519-559.
[12] READ J, PFAHRINGER B, HOLMES G, et al. Classifier chains: a review and perspectives[J]. Journal of Artificial Intelligence Research, 2021, 70: 683-718.
[13] READ J, MARTINO L, LUENGO D. Efficient Monte Carlo methods for multi-dimensional learning with classifier chains[J]. Pattern Recognition, 2014, 47(3): 1535-1546.
[14] JIA B B, ZHANG M L. Multi-dimensional classification via kNN feature augmentation[J]. Pattern Recognition, 2020, 106: 107423.
[15] WANG H B, CHEN C, LIU W W, et al. Incorporating label embedding and feature augmentation for multi-dimensional classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 6178-6185.
[16] BACH F. Breaking the curse of dimensionality with convex neural networks[J]. Journal of Machine Learning Research, 2017, 18(19): 1-53.
[17] 徐洪峰, 孙振强. 多标签学习中基于互信息的快速特征选择方法[J]. 计算机应用, 2019, 39(10): 2815-2821.
XU H F, SUN Z Q. Fast feature selection method based on mutual information in multi-label learning[J]. Journal of Computer Applications, 2019, 39(10): 2815-2821.
[18] TANG J, CHEN W H, WANG K, et al. Probability-based label enhancement for multi-dimensional classification[J]. Information Sciences, 2024, 653: 119790.
[19] PERETZ O, KOREN M, KOREN O. Naive Bayes classifier an ensemble procedure for recall and precision enrichment[J]. Engineering Applications of Artificial Intelligence, 2024, 136: 108972.
[20] DIGITALE J C, MARTIN J N, GLYMOUR M M. Tutorial on directed acyclic graphs[J]. Journal of Clinical Epidemiology, 2022, 142: 264-267.
[21] JIA B B, ZHANG M L. Maximum margin multi-dimensional classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(12): 7185-7198.
[22] LIU X Y, ZHU J H, TIAN Z Q, et al. Multi-dimensional classification via class space fusion and comprehensive label correlations[J]. Information Fusion, 2024, 111: 102521.
[23] 朱颢东, 陈宁, 李红婵. 优化的互信息特征选择方法[J]. 计算机工程与应用, 2010, 46(26): 122-124.
ZHU H D, CHEN N, LI H C. Optimized mutual information feature selection method[J]. Computer Engineering and Applications, 2010, 46(26): 122-124.
[24] HUANG L, ZHOU X Q, SHI L H, et al. Time series feature selection method based on mutual information[J]. Applied Sciences, 2024, 14(5): 1960.
[25] ZHANG Q L, LIU S P, WANG J, et al. Feature selection for multi-labeled data based on label enhancement technique and mutual information[J]. Information Sciences, 2024, 679: 121113.
[26] 贾彬彬. 基于依赖关系建模的多维分类方法研究[D]. 南京: 东南大学, 2022.
JIA B B. Research on multi-dimensional classification appr-oaches via class dependencies modeling[D]. Nanjing: Southeast University, 2022.
[27] 谢爱锋. 基于互信息的过滤式特征选择算法研究[D]. 长春: 长春工业大学, 2023.
XIE A F. Research on filter feature selection algorithm based on mutual information[D]. Changchun: Changchun University of Technology, 2023.
[28] 李占山, 杨云凯, 张家晨. 基于熵权法的过滤式特征选择算法[J]. 东北大学学报(自然科学版), 2022, 43(7): 921-929.
LI Z S, YANG Y K, ZHANG J C. Filtering feature selection algorithm based on entropy weight method[J]. Journal of Northeastern University (Natural Science), 2022, 43(7): 921-929.
[29] DEM?AR J. Statistical comparisons of classifiers over multiple data sets[J]. The Journal of Machine Learning Research, 2006, 7: 1-30.