Multi-Stage Hybrid Feature Selection Algorithm for Imbalanced Medical Data

doi:10.3778/j.issn.1002-8331.2403-0478

Abstract

Abstract: To solve the problems of high-dimensional feature and class imbalance in medical data, a multi-stage hybrid feature selection algorithm HFSIM (hybrid feature selection for imbalanced medical data) is proposed for imbalanced medical data based on the simple, fast, and effective high-dimensional feature selection algorithm SFE (simple, fast and effective high-dimensional feature selection). HFSIM algorithm adopts the improved adaptive boundary SMOTE oversampling technique to generate new minority class instances that meet the boundary conditions in order to solve the problem of class imbalance in medical data. Meanwhile, in order to solve the problem of lack of diversity in the search space, the non-selected operator rate parameter UR (unselected rate) in the SFE algorithm is optimized, which effectively avoids the problems of the algorithm’s premature convergence and its tendency to fall into the local optimum. Finally, the filtered Fisher Score method is effectively combined with the optimized UR parameter, which can obtain a better optimization capability at a lower computational cost. After experimental verification, compared with the SFE algorithm, the HFSIM algorithm achieves an accuracy of 99.67% on the Ovarian dataset, an improvement of 2.11 percentage points, and the G-means and F1 are improved by 5.13 and 2.30 percentage points respectively. In addition, by comparing the number of features and the running time, it is proved that the HFSIM algorithm guarantees the accuracy of the algorithm and reduces the computational cost.

Key words: high dimensional imbalance, feature selection, multi-stage mixing, medical data

摘要： 为解决医疗数据中存在的特征高维和类别不平衡问题，在基于简单、快速和有效高维特征选择算法SFE（simple， fast and effective high-dimensional feature selection）的基础上，提出了一种面向不平衡医疗数据的多阶段混合特征选择算法HFSIM（hybrid feature selection for imbalanced medical data）。HFSIM算法采用改进的自适应边界SMOTE过采样技术，生成符合边界条件的新少数类实例以解决医学数据中类不平衡问题。同时，为了改善搜索空间多样性不足的问题，优化了SFE算法中的非选择操作符率参数UR（unselected rate），有效避免了算法过早收敛及易陷入局部最优的问题。将过滤式Fisher Score方法与优化UR参数后的算法有效结合，使算法能以较低的计算成本获得较好寻优能力。经实验验证，相比于SFE算法，HFSIM算法在Ovarian数据集上准确率达到99.67%，提升了2.11个百分点，G-means和F1分别提升了5.13和2.30个百分点。此外，通过对比特征数量和运行时间，证明了HFSIM算法既能保证算法精度又有效降低了计算成本。

关键词: 高维不平衡, 特征选择, 多阶段混合, 医疗数据

LIU Jiaxuan, LI Daiwei, REN Lijuan, ZHANG Haiqing, CHEN Jinjing, YANG Rui. Multi-Stage Hybrid Feature Selection Algorithm for Imbalanced Medical Data[J]. Computer Engineering and Applications, 2025, 61(2): 158-169.

刘佳璇, 李代伟, 任李娟, 张海清, 陈金京, 杨瑞. 面向不平衡医疗数据的多阶段混合特征选择算法[J]. 计算机工程与应用, 2025, 61(2): 158-169.

References

[1] KOLECK T A, DREISBACH C, BOURNE P E, et al. Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review[J]. Journal of the American Medical Informatics Association, 2019, 26(4): 364-379.
[2] CAI J, LUO J, WANG S, et al. Feature selection in machine learning: a new perspective[J]. Neurocomputing, 2018, 300: 70-79.
[3] 王振飞, 袁佩瑶, 曹中亚, 等. 面向高维不平衡数据的特征选择算法[J]. 小型微型计算机系统, 2024, 45(8):1839-1846.
WANG Z F, YUAN P Y, CAO Z Y, et al. Feature selection algorithm for high dimensional unbalanced data[J]. Journal of Chinese Mini-Micro Computer Systems, 2024, 45(8): 1839-1846.
[4] ZEBARI R, ABDULAZEEZ A, ZEEBAREE D, et al. A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction[J]. Journal of Applied Science and Technology Trends, 2020, 1(1): 56-70.
[5] ZHANG Y. Safety management of civil engineering construction based on artificial intelligence and machine vision technology[J]. Advances in Civil Engineering, 2021, 2021: 1-14.
[6] SAYED G I, HASSANIEN A E, AZAR A T. Feature selection via a novel chaotic crow search algorithm[J]. Neural Computing and Applications, 2019, 31: 171-188.
[7] KAMALOV F. Orthogonal variance decomposition-based feature selection[J]. Expert Systems with Applications, 2021, 182: 115191.
[8] 孙林, 张起峰, 徐久成. 基于互信息的Fisher Score多标记特征选择[J]. 南京大学学报 (自然科学), 2023, 59(1): 55-66.
SUN L, ZHANG Q F, XU J C. Multilabel feature selection based on Fisher Score with mutual information[J]. Journal of Nanjing University (Natural Sciences), 2023, 59(1): 55-66.
[9] PALMA-MENDOZA R J, RODRIGUEZ D, DE-MARCO S L. Distributed relief-based feature selection in spark[J]. Knowledge and Information Systems, 2018, 57: 1-20.
[10] LI Y, SUN Z, LIU X, et al. Feature selection based on a large-scale many-objective evolutionary algorithm[J]. Computational Intelligence and Neuroscience, 2021(12): 1-11.
[11] SANTUCCI V, BAIOLETTI M, MILANI A. An algebraic framework for swarm and evolutionary algorithms in combinatorial optimization[J]. Swarm and Evolutionary Computation, 2020, 55: 100673.
[12] ZHANG Y, WANG S, PHILLIPS P, et al. Binary PSO with mutation operator for feature selection using decision tree applied to spam detection[J]. Knowledge-Based Systems, 2014, 64: 22-31.
[13] XUE X, YAO M, WU Z. A novel ensemble-based wrapper method for feature selection using extreme learning machine and genetic algorithm[J]. Knowledge and Information Systems, 2018, 57: 389-412.
[14] TAN P, WANG X, WANG Y. Dimensionality reduction in evolutionary algorithms-based feature selection for motor imagery brain-computer interface[J]. Swarm and Evolutionary Computation, 2020, 52: 100597.
[15] AHADZADEH B, ABDAR M, SAFARA F, et al. SFE: a simple, fast and efficient feature selection algorithm for high-dimensional data[J]. IEEE Transactions on Evolutionary Computation, 2023, 27(6):1896-1911.
[16] HANCOCK J T, WANG H, KHOSHGOFTAAR T M, et al. Data reduction techniques for highly imbalanced medicare big data[J]. Journal of Big Data, 2024, 11(1): 8.
[17] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[18] DAI Q, LIU J, LIU Y. Multi-granularity relabeled under-sampling algorithm for imbalanced data[J]. Applied Soft Computing, 2022, 124: 109083.
[19] TELIKANI A, RUDBARDEH N E, SOLEYMANPOUR S, et al. A cost-sensitive machine learning model with multitask learning for intrusion detection in IoT[J]. IEEE Transactions on Industrial Informatics, 2024, 20(3): 3880-3890.
[20] ABDULRAUF SHARIFAI G, ZAINOL Z. Feature selection for high-dimensional and imbalanced biomedical data based on robust correlation based redundancy and binary grasshopper optimization algorithm[J]. Genes, 2020, 11(7): 717.
[21] XU Z, SHEN D, NIE T, et al. A cluster-based oversampling algorithm combining SMOTE and k-means for imbalanced medical data[J]. Information Sciences, 2021, 572: 574-589.
[22] REVATHI M, RAMYACHITRA D. A modified borderline smote with noise reduction in imbalanced datasets[J]. Wireless Personal Communications, 2021, 121(3): 1659-1680.
[23] CINAR A C. A comprehensive comparison of accuracy-based fitness functions of metaheuristics for feature selection[J]. Soft Computing, 2023, 27(13): 8931-8958.
[24] DU L, XU Y, ZHU H. Feature selection for multi-class imbalanced data sets based on genetic algorithm[J]. Annals of Data Science, 2015, 2: 293-300.
[25] 苏璇, 王远军. 面向高维不平衡医学数据的特征选择算法[J]. 小型微型计算机系统, 2024, 45(2): 309-318.
SU X, WANG Y J. Feature selection algorithm for high-dimensional unbalanced medical data[J]. Journal of Chinese Mini-Micro Computer Systems, 2024, 45(2): 309-318.
[26] 于涛, 高岳林. 融入小生境和混合变异策略的鲸鱼优化算法[J]. 计算机工程与应用, 2024, 60(10):88-104.
YU T, GAO Y L. Whale optimization algorithm Integrating niche and hybrid mutation strategy[J]. Computer Engineering and Applications, 2024, 60(10): 88-104.
[27] SUN H, DU J, JIN C, et al. Global source optimization based on adaptive nonlinear particle swarm optimization algorithm for inverse lithography[J]. IEEE Photonics Journal, 2021, 13(4): 1-7.
[28] 陈博文, 邹海. 总结性自适应变异的粒子群算法[J]. 计算机工程与应用, 2022, 58(8):67-75.
CHEN B W, ZOU H. Self-conclusion and self-adaptive variation particle swarm optimization[J]. Computer Engineering and Applications, 2022, 58(8):67-75.
[29] SALESI S, COSMA G, MAVROVOUNIOTIS M. TAGA: Tabu asexual genetic algorithm embedded in a filter/filter feature selection approach for high-dimensional data[J]. Information Sciences, 2021, 565: 105-127.
[30] GAN M, ZHANG L. Iteratively local fisher score for feature selection[J]. Applied Intelligence, 2021, 51: 6167-6181.