High-Dimensional Data Feature Selection Algorithm Based on Multifactor Particle Swarm Optimization

doi:10.3778/j.issn.1002-8331.2105-0510

Abstract

Abstract:

Feature selection is an important data preprocessing technique in the field of machine learning and data mining. It aims to maximize the accuracy of classification tasks and minimize the number of optimal subset features. Using the particle swarm algorithm to find the optimal subset in the high-dimensional dataset is faced with the problems of falling into the local optimum and expensive calculations, resulting in a decrease in classification accuracy. To solve this problem, a high-dimensional data feature selection algorithm based on multifactor particle swarm optimization is proposed. Firstly, the evolutionary multi-task algorithm framework is introduced, and a two-task model generation strategy is proposed, which strengthens population communication through knowledge transfer between tasks and improves population diversity to improve the shortcomings that tend to fall into local optimum. Secondly, the design is based on the initial strategy of sparse representation, the initial solution with sparse representation is designed in the initial stage of the algorithm, which reduces the computational cost of the population when it tends to the optimal solution set. The experimental results on 6 public medical high-dimensional datasets show that the proposed algorithm can effectively achieve the classification task and obtain better accuracy.

Key words: high-dimensional data, feature selection, evolutionary multitasking, Particle Swarm Optimization（PSO）

摘要：

特征选择是机器学习和数据挖掘领域中一项重要的数据预处理技术，它旨在最大化分类任务的精度和最小化最优子集特征个数。运用粒子群算法在高维数据集中寻找最优子集面临着陷入局部最优和计算代价昂贵的问题，导致分类精度下降。针对此问题，提出了基于多因子粒子群算法的高维数据特征选择算法。引入了进化多任务的算法框架，提出了一种两任务模型生成的策略，通过任务间的知识迁移加强种群交流，提高种群多样性以改善易陷入局部最优的缺陷；设计了基于稀疏表示的初始化策略，在算法初始阶段设计具有稀疏表示的初始解，降低了种群在趋向最优解集时的计算开销。在6个公开医学高维数据集上的实验结果表明，所提算法能够有效实现分类任务且得到较好的精度。

关键词: 高维数据, 特征选择, 进化多任务, 粒子群算法（PSO）

LIN Weixing, WANG Yujia, CHEN Wanfen, LIANG Haina. High-Dimensional Data Feature Selection Algorithm Based on Multifactor Particle Swarm Optimization[J]. Computer Engineering and Applications, 2021, 57(22): 199-207.

林炜星，王宇嘉，陈万芬，梁海娜. 基于多因子粒子群的高维数据特征选择算法[J]. 计算机工程与应用, 2021, 57(22): 199-207.

References

[1] ADELI A，BROUMANDNIA A.Image steganalysis using improved particle swarm optimization based feature selection[J].Applied Intelligence，2018，48（6）：1609-1622.
[2] QARAAD M，AMJAD S，FATHI H，et al.Feature selection techniques for cancer classification applied to microarray data：a survey[C]//2019 International Conference on Intelligent Systems and Advanced Computing Sciences，2019：26-27.
[3] 梁家铭，赵洁，郑鹏，等.特征选择下融合图像和文本分析的在线短租平台信任计算框架[J].数据分析与知识发现，2021，5（2）：129-140.
LIANG J M，ZHAO J，ZHENG P，et al.Framework for computing trust in online short-rent platform using feature selection of images and texts[J].Data Analysis and Knowledge Discovery，2021，5（2）：129-140.
[4] 萧秋兰，郑虹.微阵列肿瘤分类混合基因选择算法[J].长春工业大学学报，2019，40（6）：26-32.
XIAO Q L，ZHENG H.A hybrid gene selection algorithm for microarray cancer classification[J].Journal of Chang-
chun University of Technology，2019，40（6）：26-32.
[5] BRANKOVIC A，FALSONE A，PRANDINI M，et al.A feature selection and classification algorithm based on randomized extraction of model populations[J].IEEE Transactions on Cybernetics，2018，48（4）：1151-1162.
[6] KIRA K，RENDELL L A.The feature selection problem：traditional methods and a new algorithm[C]//10th National Conference on Artificial Intellignce，San Jose，1992：129-134.
[7] LIU H，SETIONO R.Feature selection and classification—a probabilistic wrapper approach[C]//9th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert System，Fukuoka，1996：419-424.
[8] TRAN B，ZHANG M，BING X.A PSO based hybrid feature selection algorithm for high-dimensional classification[C]//2016 IEEE Congress on Evolutionary Computation，2016：24-29.
[9] HUANG Z，YANG C，ZHOU X，et al.A hybrid feature selection method based on binary state transition algorithm and ReliefF[J].IEEE Journal of Biomedical and Health Informatics，2019，23（5）：1888-1898.
[10] MAFARJA M M，SEYEDALI M.Hybrid binary ant lion optimizer with rough set and approximate entropy reducts for feature selection[J].Soft Computing，2019，23：6249-6265.
[11] KENNEDY J，EBERHART R.Particle swarm optimization[C]//1995 International Conference on Neural Networks，1995.
[12] 李炜，巢秀琴.改进的粒子群算法优化的特征选择方法[J].计算机科学与探索，2019，13（6）：990-1004.
LI W，CHAO X Q.Improved particle swarm optimization method for feature selection[J].Journal of Frontiers of Computer Science and Technology，2019，13（6）：990-1004.
[13] 王金杰，李炜.混合互信息和粒子群算法的多目标特征选择方法[J].计算机科学与探索，2020，14（1）：83-95.
WANG J J，LI W.Multi-objective feature selection method based on hybrid MI and PSO algorithm[J].Journal of Frontiers of Computer Science and Technology，2020，14（1）：83-95.
[14] LANE M C，XUE B，LIU I，et al.Gaussian based particle swarm optimisation and statistical clustering for feature selection[C]//2014 European Conference on Evolutionary Computation in Combinational Optimization.Cham：Springer，2014：133-144.
[15] TRAN B，XUE B，ZHANG M.Variable-length particle swarm optimization for feature selection on high-dimensional classification[J].IEEE Transactions on Evolutionary Computation，2019，23（3）：473-487.
[16] BANKA H，DARA S.A Hamming distance based binary particle swarm optimization（HDBPSO） algorithm for high dimensional feature selection，classification and validation[J].Pattern Recognition Letters，2015，52（15）：94-100.
[17] 代永强，郭小燕，王敏，等.基于混合蛙跳算法的高维生物医学数据特征选择方法[J].计算机应用研究，2021，38（4）：1062-1068.
DAI Y Q，GUO X Y，WANG M，et al.Feature selection of high-dimensional biomedical data based on shuffled frog leaping algorithm[J].Application Research of Computers，2021，38（4）：1062-1068.
[18] 王俊，冯军，张戈，等.基于改进灰狼优化算法的医学数据特征选择应用研究[J].河南大学学报（自然科学版），2020，50（5）：570-578.
WANG J，FENG J，ZHANG G，et al.Application research of medical data feature selection based on improved grey wolf optimization algorithm[J].Journal of Henan University（Natural Science），2020，50（5）：570-578.
[19] GUPTA A，ONG Y S，FENG L，et al.Multiobjective multifactorial optimization in evolutionary multitasking[J].IEEE Transactions on Cybernetics，2016，47（7）：1-14.
[20] GUPTA A，ONG Y S，FENG L.Multifactorial evolution：toward evolutionary multitasking[J].IEEE Transactions on Evolutionary Computation，2016，20（3）：343-357.
[21] FENG L，ZHOU W，ZHOU L，et al.An empirical study of multifactorial PSO and multifactorial DE[C]//2017 IEEE Congress on Evolutionary Computation，2017：5-8.
[22] TIAN Y，ZHANG X，WANG C，et al.An evolutionary algorithm for large-scale sparse multiobjective optimization problems[J].IEEE Transactions on Evolutionary Computation，2020，24（2）：380-393.
[23] JOLLIFFE I T.Principal component analysis[J].Journal of Marketing Research，2002，87（4）：513.
[24] NG Y.ICA with reconstruction cost for efficient overcomplete feature learning[J].Advances in Neural Information Processing Systems，2015，12（11）：1017-1025.