Research on Feature Selection of Multi-Objective Optimization

doi:10.3778/j.issn.1002-8331.2206-0230

Abstract

Abstract: Feature selection is an effective dimensionality reduction method in the field of pattern recognition. When multiple objectives involved in feature selection conflict with each other and are difficult to balance, it is a research hotspot to regard feature selection as a multi-objective optimization problem. In order to facilitate researchers to systematically understand the research status and development trend in the field of multi-objective feature selection, this paper reviews the methods of multi-objective feature selection. This paper clarifies the nature of feature selection and multi-objective optimization. According to the differences and characteristics of multi-objective optimization methods, it focuses on the advantages and disadvantages of various multi-objective optimization feature selection methods. It discusses the existing multi-objective optimization feature selection research methods problems and prospects for the future.

Key words: feature selection, multi-objective optimization, evolutionary algorithms

摘要： 特征选择是模式识别领域中有效的降维方法，当特征选择涉及到的多个目标彼此冲突，难以平衡时，将特征选择视为多目标优化问题是时下的研究热点。为方便研究者系统地了解多目标特征选择领域的研究现状和发展趋势，对多目标特征选择方法进行综述。阐明了特征选择和多目标优化的本质；根据多目标优化方法的区别和特点，重点对比剖析各类多目标优化特征选择方法的优劣势；讨论现有多目标优化特征选择研究方法存在的问题以及对未来的展望。

关键词: 特征选择, 多目标优化, 进化算法

ZHANG Mengting, DU Jianqiang, LUO Jigen, NIE Bin, XIONG Wangping, LIU Ming, ZHAO Shuhan. Research on Feature Selection of Multi-Objective Optimization[J]. Computer Engineering and Applications, 2023, 59(3): 23-32.

张梦婷, 杜建强, 罗计根, 聂斌, 熊旺平, 刘明, 赵书含. 多目标优化特征选择研究综述[J]. 计算机工程与应用, 2023, 59(3): 23-32.

References

[1] SHINADA N K，NAOKI K，MEGUMI I，et al.Optimizing machine-learning models for mutagenicity prediction through better feature selection[J].Mutagenesis，2022，37（3/4）：191-202.
[2] 肖进胜，张舒豪，陈云华，等.双向特征融合与特征选择的遥感影像目标检测[J].电子学报，2022，50（2）：267-272.
XIAO J S，ZHANG S H，CHEN Y H，et al.Remote sensing image object detection based on bidirectional feature fusion and feature selection[J].Acta Electronica Sinica，2022，50（2）：267-272.
[3] 武炜杰，张景祥.融合分类信息的随机森林特征选择算法及应用[J].计算机工程与应用，2021，57（17）：147-156.
WU W J，ZHANG J X.Random forest feature selection algorithm based on categorization information and application[J].Computer Engineering and Applications，2021，57（17）：147-156.
[4] 李博.基于网络生物标志物的特征选择算法及其在癌症检测上的应用[D].长春：吉林大学，2020.
LI B.Feature selection algorithm based on network biomarkers and its application in cancer detection[D].Changchun：Jilin University，2020.
[5] 王家琪，张莉.面向文本特征选择的去冗余相对判别准则[J].山西大学学报（自然科学版），2021，44（4）：688-694.
WANG J Q，ZHANG L.De-redundantly relative discrimination criterion for text feature selection[J].Journal of Shanxi University（Natural Science Edition），2021，44（4）：688-694.
[6] 戴忠华，周穗华，张晓兵.多目标优化的舰船磁场建模方法[J].物理学报，2021，70（16）：147-159.
DAI Z H，ZHOU S H，ZHANG X B.Multi-objective optimization of ship magnetic field modeling method[J].Acta Physica Sinica，2021，70（16）：147-159.
[7] 高世达，薄翠梅，李俊，等.工业有机污染物治理过程的多目标优化控制[J].控制理论与应用，2022，39（1）：187-199.
GAO S D，BO C M，LI J，et al.Multi-objective optimization control of industrial organic pollutants treatment process[J].Control Theory ＆ Applications，2022，39（1）：187-199.
[8] 周文婷，司玉鹏，何宏舟，等.基于量子多目标优化算法的回焊炉炉温设计[J].焊接学报，2022，43（1）：85-91.
ZHOU W T，SI Y P，HE H Z，et al.Design of reflow oven furnace temperature based on quantum multi-objective optimization algorithm[J].Transactions of the China Welding Institution，2022，43（1）：85-91.
[9] 李郅琴，杜建强，聂斌，等.特征选择方法综述[J].计算机工程与应用，2019，55（24）：10-19.
LI Z Q，DU J Q，NIE B，et al.Summary of feature selection methods[J].Computer Engineering and Applications，2019，55（24）：10-19.
[10] Al-TASHI Q，ABDULKADIR S J，RAIS H，et al.Approaches to multi-objective feature selection：a systematic literature review[J].IEEE Access，2020，8：125076-125096.
[11] 许尧.过滤式特征选择算法研究[D].合肥：合肥工业大学，2015.
XU Y.Research on filtering feature selection algorithm[D].Hefei：Hefei University of Technology，2015.
[12] 武小军，周文心，董永新.一种改进的嵌入式特征选择算法及应用[J].同济大学学报（自然科学版），2022，50（2）：153-159.
WU X J，ZHOU W X，DONG Y X.A novel embedded feature selection algorithm and its application[J].Journal of Tongji University（Natural Science），2022，50（2）：153-159.
[13] ROUHANI H，FATHABADI A，BAARTMAN J.A wrapper feature selection approach for efficient modelling of gully erosion susceptibility mapping[J].Progress in Physical Geography，2021，45（4）.
[14] WOEGINGER G J.Exact algorithms for NP-hard problems：a survey[C]//5th International Workshop on Combinatorial Optimization，2003.
[15] LIU H， YU L.Toward integrating feature selection algorithms for classification and clustering[J].IEEE Transactions on Knowledge and Data Engineering，2005，17（4）：491-502.
[16] 尚正阳，顾寄南，唐仕喜，等.针对几种元启发式算法的应用性能对比研究[J].机械设计与制造，2021（4）：34-38.
SHANG Z Y，GU J N，TANG S X，et al.Comparative study on application performance of several meta-heuristic algorithms[J].Machinery Design & Manufacture，2021（4）：34-38.
[17] NEWMAN M E J.Power laws，Pareto distributions and Zipf’slaw[J].Contemporary Physics：A Review of Physics and Associated Technologies，2005，46（5）：323-351.
[18] BRANKE J，DEB K，DIEROLF H，et al.Finding knees in multi-objective optimization[C]//8th International Conference on Parallel Problem Solving from Nature，2004.
[19] MARLER R T，ARORA J S.Survey of multi-objective optimization methods for engineering[J].Structural and Multidisciplinary Optimization，2004，26（6）.
[20] 云慧敏，吴锋，殷勇，等.基于线性加权评价法的碟式斯特林机多目标优化[J].武汉工程大学学报，2020，42（2）：218-223.
YUN H M，WU F，YIN Y，et al.Multi-objective optimization of dish stirling engine based on linear weighted evaluation function method[J].Journal of Wuhan Institute of Technology，2020，42（2）：218-223.
[21] 张亚辉，胡小锋，吴传珣.基于ε-约束法的多目标双边装配线再平衡问题[J].计算机集成制造系统，2016，22（11）：2551-2562.
ZHANG Y H，HU X F，WU C X.Multi-objective two-sided assembly line rebalancing problem based on ε-constraint method[J].Computer Integrated Manufacturing Systems，2016，22（11）：2551-2562.
[22] TSENG C H，LU T W.Minimax multi-objective optimization in structural design[J].International Journal for Numerical Methods in Engineering，2010，30（6）：1213-1228.
[23] SCHAFFER J D.Multiple objective optimization with vector evaluated genetic algorithms[C]//Proceedings of the 1st International Conference on Genetic Algorithms，Pittsburgh，PA，USA，July 1985.Hillsdale：Lawrence Erlbaum Associates Publishers，1985.
[24] WANG S C，CHEN T C.Using NSGA-II to solve multi-objective competitive location problem with cooperative coverage for distance-based attractiveness[J].Journal of Intelligent and Fuzzy Systems，2021（153）：1-12.
[25] 冯茜，李擎，全威，等.多目标粒子群优化算法研究综述[J].工程科学学报，2021，43（6）：745-753.
FENG Q，LI Q，QUAN W，et al.Overview of multiobjective particle swarm optimization algorithm[J].Chinese Journal of Engineering，2021，43（6）：745-753.
[26] 刁兴春，刘艺，曹建军，等.多目标蚁群优化研究综述[J].计算机科学，2017，44（10）：7-13.
DIAO X C，LIU Y，CAO J J，et al.Reviews of multiobjective ant colony optimization[J].Computer Science，2017，44（10）：7-13.
[27] 颜景斌，沈云森，刘思，等.SWISS整流器多目标优化[J].哈尔滨理工大学学报，2021，26（1）：86-92.
YAN J B，SHEN Y S，LIU S，et al.Multi-objective optimization of SWISS rectifier[J].Journal of Harbin University of Science and Technology，2021，26（1）：86-92.
[28] 李琳，应时，董波.一种求解面向服务软件部署优化问题的多目标蚁群算法[J].中南大学学报（自然科学版），2017，48（9）：2376-2387.
LI L，YING S，DONG B.A multi-objective ant colony algorithm for deployment optimization of service-oriented application[J].Journal of Central South University（Science and Technology），2017，48（9）：2376-2387.
[29] 王丽萍，任宇，邱启仓，等.多目标进化算法性能评价指标研究综述[J].计算机学报，2021，44（8）：1590-1619.
WANG L P，REN Y，QIU Q C，et al.Survey on performance indicators for multi-objective evolutionary algorithms[J].Chinese Journal of Computers，2021，44（8）：1590-1619.
[30] VELDHUIZEN D A V，LAMONT G B.Evolutionary computation and convergence to a pareto front[D].Stanford University California，1999.
[31] TSOU C S，FANG H H，CHANG H H，et al.An improved particle swarm pareto optimizer with local search and clustering[C]//Simulated Evolution and Learning，2006：400-407.
[32] SCHOTT J R.Fault tolerant design using single and multi-criteria genetic algorithms[D].Massachusetts Institute of Technology，1995.
[33] COELLO C.Solving multi-objective optimization problems using an artificial immune system[J].Genetic Programming and Evolvable Machines，2005，6.
[34] ZITZLER E，THIELE L，LAUMANNS M，et al.Performance assessment of multi-objective optimizers：an analysis and review[J].IEEE Transactions on Evolutionary Computation，2003，7（2）：117-132.
[35] ZITZLER E，THIELE L.Multi-objective evolutionary algorithms：a comparative case study and the strength Pareto approach[J].IEEE Transactions on Evolutionary Computation，1999，3（4）：257-271.
[36] CHUANG L Y，CHANG H W，TU C J，et al.Improved binary PSO for feature selection using gene expression data[J].Computational Biology & Chemistry，2008，32（1）：29-38.
[37] TRAN B，XUE B，ZHANG M.A new representation in PSO for discretization-based feature selection[J].IEEE Transactions on Cybernetics，2017：1-14.
[38] GOT A，MOUSSAOUI A，ZOUACHE D.Hybrid filter-wrapper feature selection using whale optimization algorithm：a multi-objective approach[J].Expert Systems with Applications，2021，183：115312.
[39] WANG X H，ZHANG Y，SUN X Y，et al.Multi-objective feature selection based on artificial bee colony：an acceleration approach with variable sample size[J].Applied Soft Computing，2020，88.
[40] 万红，李蒙蒙，王昊锋，等.多目标优化在特征选择子集评价中的应用[J].计算机应用研究，2020，37（8）：2320-2323.
WAN H，LI M M，WANG H F，et al.Application of multi-objective optimization in feature selection subset evaluation[J].Application Research of Computers，2020，37（8）：2320-2323.
[41] USMAN A M，YUSOF U K，NAIM S.Filter-based multi-objective feature selection using NSGA III and cuckoo optimization algorithm[J].IEEE Access，2020，8：76333-76356.
[42] LABANI M，MORADI P，JALILI M.A multi-objective genetic algorithm for text feature selection using the relative discriminative criterion[J].Expert Systems with Applications，2020，149：113276.
[43] NAYAK S K，ROUT P K，JAGADEV A K，et al.Elitism based multi-objective differential evolution for feature selection：a filter approach with an efficient redundancy measure[J].Journal of King Saud University-Computer and Information Sciences，2017，32（2）：174-187.
[44] XUE B，CERVANTE L，SHANG L，et al.A multi-objective particle swarm optimization for filter-based feature selection in classification problems[J].Connection Science，2012，24（2/3）：91-116.
[45] XUE B，ZHANG M，BROWNE W N.Particle swarm optimization for feature selection in classification：a multi-objective approach[J].IEEE Transactions on Cybernetics，2013，43（6）：1656-1671.
[46] 李敏，章国豪，陈梓樑，等.基于差分进化的多目标粒子群特征选择算法[J].计算机应用研究，2020，37（1）：76-79.
LI M，ZHANG G H，CHEN Z L，et al.Multi-objective particle swarm optimization algorithm using differential evolution for feature selection[J].Application Research of Computers，2020，37（1）：76-79.
[47] NOURI-MOGHADDAM B，GHAZANFARI M，FATHIAN M.A novel multi-objective forest optimization algorithm for wrapper feature selection[J].Expert Systems with Applications，2021，175（3）：114737.
[48] AL-TASHI Q，ABDULKADIR S J，RAIS H，et al.Binary multi-objective grey wolf optimizer for feature selection in classification[J].IEEE Access，2020，8：106247-106263.
[49] DONG H，SUN J，SUN X，et al.A many-objective feature selection for multi-label classification[J].Knowledge-Based Systems，2020，208（7）：106456.
[50] FARIS H，HEIDARI A A，MAFARJA M M，et al.A robust multi-objective feature selection model based on local neighborhood multi-verse optimization[J].IEEE Access，2021，9：100009-100028.
[51] GONZALEZ J，ORTEGA J，DAMAS M，et al.A new multi-objective wrapper method for feature selection—accuracy and stability analysis for BCI[J].Neurocomputing，2019，333：407-418.
[52] KOZODOI N，LESSMANN S，PAPAKONSTANTINOU K，et al.A multi-objective approach for profit-driven feature selection in credit scoring[J].Decision Support Systems，2019，120：106-117.
[53] SOHRABI M K，TAJIK A.Multi-objective feature selection for warfarin dose prediction[J].Computational Biology & Chemistry，2017，69：126-133.
[54] XUE Y，TANG Y，XU X，et al.Multi-objective feature selection with missing data in classification[J].IEEE Transactions on Emerging Topics in Computational Intelligence，2022，6（2）：355-364.
[55] 王金杰，李炜.混合互信息和粒子群算法的多目标特征选择方法[J].计算机科学与探索，2020，14（1）：83-95.
WANG J J，LI W.Multi-objective feature selection method based on hybrid MI and PSO algorithm[J].Journal of Frontiers of Computer Science and Technology，2020，14（1）：83-95.
[56] 陈陆爽，周晖.慢性肾病预测的多目标特征选择[J].计算机工程与设计，2022，43（3）：698-705.
CHEN L S，ZHOU H.Multi-objective feature selection for chronic kidney disease prediction[J].Computer Engineering and Design，2022，43（3）：698-705.
[57] 刘艺，曹建军，刁兴春，等.稳定特征选择的多目标蚁群优化[J].国防科技大学学报，2018，40（6）：118-123.
LIU Y，CAO J J，DIAO X C，et al.Multiobjective ant colony optimization for stable feature selection[J].Journal of National University of Defense Technology，2018，40（6）：118-123.
[58] YU Z，JK A，HG C.Many-objective optimization of feature selection based on two-level particle cooperation[J].Information Sciences，2020，532：91-109.
[59] LIU Z，CHANG B，CHENG F.An interactive filter-wrapper multi-objective evolutionary algorithm for feature selection[J].Swarm and Evolutionary Computation，2021（4）：100925.
[60] MARWA H，SLIM B，CHIH-CHENG H，et al.A multi-objective hybrid filter-wrapper evolutionary approach for feature selection[J].Memetic Computing，2019，11（2）.
[61] SHARMA A，RINKLE R.C-HMOSHSSA：gene selection for cancer classification using multi-objective meta-heuristic and machine learning methods—sciencedirect[J].Computer Methods and Programs in Biomedicine，2019，178：219-235.