Heterogeneous Ensemble Surrogate Assisted Multi-objective Particle Swarm Optimization Algorithm

doi:10.3778/j.issn.1002-8331.2012-0010

Abstract

Abstract:

Aiming at the problem that the surrogate-assisted evolutionary algorithm is difficult to construct a high-quality surrogate model from a small number of sample points when reducing expensive fitness evaluation, a heterogeneous ensemble surrogate-assisted multi-objective particle swarm optimization algorithm is proposed. This method uses the weighted average method to combine the Kriging model and the radial basis function network model into a high-precision heterogeneous integrated model to achieve the purpose of enhancing the algorithm’s ability to process uncertain information. Two surrogate models based on ensemble learning are applied to global search and local search respectively. Based on the framework of multi-objective particle swarm optimization algorithm, the newly proposed method adaptively constructs a heterogeneous ensemble model for each objective function, and then uses the non-dominated solution of the model to guide the update of the particle swarm and obtains the optimal solution set of the objective function. Experimental results show that the proposed method improves the search capability of the surrogate model, reduces the number of evaluations, and as the search dimension increases, its computational complexity also has better scalability.

Key words: multi-objective optimization, particle swarm optimization algorithm, Kriging model, radial basis function network model, heterogeneous ensemble

摘要：

针对代理辅助进化算法在减少昂贵适应度评估时难以通过少量样本点构造高质量代理模型的问题，提出异构集成代理辅助多目标粒子群优化算法。该方法通过使用加权平均法将Kriging模型和径向基函数网络模型组合成高精度的异构集成模型，达到增强算法处理不确定性信息能力的目的。基于集成学习的两种代理模型分别应用于全局搜索和局部搜索，在多目标粒子群优化算法框架基础上，新提出的方法为每个目标函数自适应地构造了异构集成模型，利用其模型的非支配解来指导粒子群的更新，得出目标函数的最优解集。实验结果表明，所提方法提高了代理模型的搜索能力，减少了评估次数，并且随着搜索维度的增加，其计算复杂性也具有更好的可扩展性。

关键词: 多目标优化, 粒子群优化算法, Kriging模型, 径向基函数网络模型, 异构集成

CHEN Wanfen, WANG Yujia, LIN Weixing. Heterogeneous Ensemble Surrogate Assisted Multi-objective Particle Swarm Optimization Algorithm[J]. Computer Engineering and Applications, 2021, 57(23): 71-80.

陈万芬，王宇嘉，林炜星. 异构集成代理辅助多目标粒子群优化算法[J]. 计算机工程与应用, 2021, 57(23): 71-80.

References

[1] HU J，ZHOU Q，JIANG P，et al.An adaptive sampling method for variable-fidelity surrogate models using improved hierarchical kriging[J].Engineering Optimization，2017（5）：1-19.
[2] BUHMANN M D.Radial basis functions：theory and implementations[M].Cambridge，UK：Cambridge University Press，2003.
[3] MYERS R H，MONTGOMERY D C.Response surface methodology：process and product optimization using designed experiments[M].New York：Wiley，1995.
[4] STEIN M L.Interpolation of spatial data：some theory for kriging[M].Berlin：Springer-Verlag，1999.
[5] GUNN S R.Support vector machines for classification and regression[R].Southampton：Southampton University of Southampton.Image Speech and Intelligent Systems Research Group，1997.
[6] GOEL T，HAFTKA R T，SHYY W，et al.Ensemble of surrogates[J].Structural and Multi-disciplinary Optimization，2007，33（3）：199-216.
[7] DONG H，SONG B，WANG P，et al.Hybrid surrogate-based optimization using space reduction（HSOSR） for expensive black-box functions[J].Applied Soft Computing，2018，64：641-655.
[8] YE P，PAN G，DONG Z.Ensemble of surrogate based global optimization methods using hierarchical design space reduction[J].Structural and Multi-disciplinary Optimization，2018：1-18.
[9] SINGH H K，RAY T，SMITH W.Surrogate assisted simulated annealing（SASA） for constrained multi-objective optimization[C]//Proceedings of the IEEE Congress on Evolutionary Computation，2010：1-8.
[10] LIM D，JIN Y.Generalizing surrogate-assisted evolutionary computation[J].IEEE Transactions on Evolutionary Computation，2010，14：329-354.
[11] MARTINEZ S Z，COELLO C A C.MOEA/D assisted by RBF networks for expensive multi-objective optimization problems[C]//Proceedings of the Genetic and Evolutionary Computation Conference，2013：1405-1412.
[12] ROSALES-PéREZ A，COELLO C A C，GONZALEZ J A，et al.A hybrid surrogate-based approach for evolutionary multi-objective optimization[C]//Proc IEEE Congress on Evolutionary Computation，Cancun，Mexico，2013：2548-2555.
[13] WANG H，JIN Y，DOHERTY J.Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems[J].IEEE Transactions on Cybernetics，2017，47（9）：2664-2677.
[14] WANG Y，YIN D Q，YANG S.Global and local surrogate-assisted differential evolution for expensive constrained optimization[J].IEEE Transactions on Cybernetics，2018，99：1-15.
[15] WANG H，JIN Y，JANSON J O.Data-driven surrogate-assisted multi-objective evolutionary optimization of a trauma system[J].IEEE Transactions on Evolutionary Computation，2016，20（6）：939-952.
[16] WANG H，JIN Y，SUN C，et al.Offline data-driven evolutionary optimization using selective surrogate ensembles[J].IEEE Transactions on Evolutionary Computation，2019，23：203-216.
[17] AWAD N H，ALI M Z，MALLIPEDDI R.An improved differential evolution algorithm using efficient adapted surrogate model for numerical optimization[J].Information Sciences，2018：326-347.
[18] CAI X W，GAO L，LI X Y，et al.Surrogate-guided differential evolution algorithm for high dimensional expensive problems[J].Swarm and Evolutionary Computation，2019，48：288-311.
[19] EBERHART R，KENNEDY J.A new optimizer using particle swarm theory[C]//Proceedings of the Sixth International Symposium on Micro Machine and Human Science，1995：39-43.
[20] 余伟伟，谢承旺.一种多策略混合的粒子群优化算法[J].计算机科学，2018，45（S1）：120-123.
YU Weiwei，XIE Chengwang.A multi-strategy hybrid particle swarm optimization algorithm[J].Computer Science，2018，45（S1）：120-123.
[21] 周蓉，李俊，王浩.基于灰狼优化的反向学习粒子群算法[J].计算机工程与应用，2020，56（7）：48-56.
ZHOU Rong，LI Jun，WANG Hao.Reverse learning particle swarm optimization based on grey wolf optimization[J].Computer Engineering and Applications，2020，56（7）：48-56.
[22] 邓志诚，孙辉，赵嘉，等.具有动态子空间的随机单维变异粒子群算法[J].计算机科学与探索，2020，14（8）：1409-1426.
DENG Zhicheng，SUN Hui，ZHAO Jia，et al.Stochastic single-dimensional mutated particle swarm optimization with dynamic subspace[J].Journal of Frontiers of Computer Science and Technology，2020，14（8）：1409-1426.
[23] 魏锋涛，卢凤仪.融合核函数在改进径向基代理模型中的应用[J].计算机工程与应用，2019，55（7）：58-65.
WEI Fengtao，LU Fengyi.Application of hybrid kernel function in improved radial basis function meta-model[J].Computer Engineering and Applications，2019，55（7）：58-65.
[24] DíAZ-MANRíQUEZ A，TOSCANO G，COELLO C A.Comparison of metamodeling techniques in evolutionary algorithms[J].Soft Computing，2017，21：5647-5663.
[25] PARK J S.Optimal Latin-hypercube designs for computer experiments[J].Journal of Statistical Planning and Inference，1994，39（1）：95-111.
[26] 周志华.机器学习[M].北京：清华大学出版社，2016.
ZHOU Zhihua.Machine learning[M].Beijing：Tsinghua University Press，2016.
[27] BROWN G，WYATT J L，T?NO P.Managing diversity in regression ensembles[J].J Mach Learn Res，2005，6：1621-1650.
[28] ZITZLER E，DEB K，THIELE L.Comparison of multi-objective evolutionary algorithms：empirical results[J].Evolutionary Computation，2000，8（2）：173-195.
[29] DEB K，PRATAP A，AGARWAL S，et al.A fast and elitist multi-objective genetic algorithm：NSGA-II[J].IEEE Transactions on Evolutionary Computation，2002，6（2）：182-197.
[30] SIERRA M R，COELLO C A C.Improving PSO-based multi-objective optimization using crowding，mutation and ?-dominance evolutionary multi-criterion optimization[C]//International Conference on Evolutionary Multi-Criterion Optimization，2005：505-519.
[31] 王刚成，马宁，顾解忡.基于Kriging代理模型的船舶水动力性能多目标快速协同优化[J].上海交通大学学报，2018，52（6）：666-673.
WANG Gangcheng，MA Ning，GU Xiechong.Fast collaborative multi-objective optimization for hydrodynamic based on Kriging surrogate model[J].Journal of Shanghai Jiaotong University，2018，52（6）：666-673.
[32] JIE H，WU Y，ZHAO J.et al.An efficient multi-objective PSO algorithm assisted by Kriging meta-model for expensive black-box problems[J].J Glob Optimization，2017，67：399-423.
[33] VAN VELDHUIZEN D A，LAMONT G B.Multi-objective evolutionary algorithm research：a history and analysis[J].Evolutionary Computation，1998，8（2）：125-147.
[34] SCHOTT J R.Fault tolerant design using single and multi-criteria genetic algorithm optimization[D].Massachusetts Institute of Technology，1995.
[35] WHILE L，HINGSTON P，BARONE L，et al.A faster algorithm for calculating hyper-volume[J].IEEE Transactions on Evolutionary Computation，2006，10（1）：29-38.