贝叶斯优化的XGBoost信用风险评估模型

doi:10.3778/j.issn.1002-8331.2303-0422

摘要/Abstract

摘要： 信用风险评估是银行和其他金融机构信贷审批业务中必不可少的一环。为进一步提高信用风险评估的准确率和模型可解释性，提出了基于贝叶斯优化的极端梯度提升树（extreme gradient boosting，XGBoost）信用风险评估模型。XGBoost集成学习模型预测准确率高，基学习器采用树模型，易于可视化，具有良好的可解释性。然而，XGBoost模型超参数众多，模型效果依赖于超参数的精确设置。在这项研究中，采用贝叶斯高斯过程（GP）作为XGBoost的超参数优化器，并与网格搜索、随机搜索进行比较研究。所提出的模型在三个信用贷款数据集上进行训练和测试，选择准确率和F1分数等四项指标评价模型性能。实验结果发现将贝叶斯高斯过程用于XGBoost模型的超参数优化，收敛速度快。所提出的模型在三个数据集上的准确率比表现最好的对比模型分别高出3.5、3.62和0.91个百分点。

关键词: 信用风险评估, 贝叶斯优化, 高斯过程, XGBoost模型

Abstract: Credit scoring is the core operation of credit granting for banks or lending establishments. To further improve the accuracy and interpretability of credit scoring, an extreme gradient boosting（XGBoost） credit scoring model based Bayesian optimization is proposed. XGBoost ensemble learning method is based on decision tree, which is easy to visualized and interpretable. However, XGBoost has large amount of hyper-parameters and the model performance depends on the exact setting of hyper-parameters. In this study, Bayesian Gaussian processes（GP） is used as the hyper-parameter optimizer for XGBoost and is compared with grid search and random search. Experiments are performed with three credit datasets to train and test the proposed model. Four metrics such as accuracy and F1-score are used to evaluate the model performance. The experimental results show that Bayesian Gaussian processes converges fast as the optimizer. The accuracy of the proposed model on the three datasets is 3.5, 3.62, and 0.91 percentage points higher than the best performing comparison model, respectively.

Key words: credit scoring, Bayesian optimization, Gaussian processes, XGBoost model

贾颖, 赵峰, 李博, 葛诗煜. 贝叶斯优化的XGBoost信用风险评估模型[J]. 计算机工程与应用, 2023, 59(20): 283-294.

JIA Ying, ZHAO Feng, LI Bo, GE Shiyu. XGBoost Optimized by Bayesian Optimization for Credit Scoring[J]. Computer Engineering and Applications, 2023, 59(20): 283-294.

参考文献

[1] 操玮，李灿，贺婷婷，等.基于集成学习的中国P2P网络贷款信用风险预警模型的对比研究[J].数据分析与知识发现，2018，22（10）：65-76.
CAO W，LI C，HE T T，et al.Predicting credit risks of P2P loans in China based on ensemble learning methods[J].Data Analysis and Knowledge Discovery，2018，22（10）：65-76.
[2] BAHNSEN A C，AOUADA D，OTTERSTEN B.Example-dependent cost-sensitive logistic regression for credit scoring[C]//International Conference on Machine Learning & Applications，2015.
[3] WEST D.Neural network credit scoring models[J].Computers & Operations Research，2000，27（11/12）：1131-1152.
[4] 王静.基于BP神经网络和线性判别分析的信用评分模型研究[D].洛阳：河南科技大学，2022.
WANG J.Research on credit score model based on BP neural network and linear discriminant analysis[D].Luo-
yang：Henan University of Science and Technology，2022.
[5] HUANG C L，CHEN M C，WANG C J.Credit scoring with a data mining approach based on support vector machines[J].Expert Systems with Applications，2007，33（4）：847-856.
[6] NING C，RIBEIRO B，AN C.Financial credit risk assessment：a recent review[J].Artificial Intelligence Review，2016，45（1）：1-23.
[7] KAO L J，CHIU C C，CHIU F Y.A Bayesian latent variable model with classification and regression tree approach for behavior and credit scoring[J].Knowledge-Based Systems，2012：245-252.
[8] QIAN H Y，MA P，GAO S F，et al.Soft reordering one-dimensional convolutional neural network for credit scoring[J].Knowledge-Based Systems，2023，266：110414.
[9] GICI? A，?ONKO D，SUBASI A.Intelligent credit scoring using deep learning methods[J].Concurrency and Computation：Practice and Experience，2023，35：7637.
[10] XIA Y F，LIU C Z，LI Y Y，et al.A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring[J].Expert Systems with Applications，2017，78：225-241.
[11] LESSMANN S，BAESENS B，SEOW H，et al.Benchmarking state-of-the-art classification algorithms for credit scoring：an update of research[J].European Journal of Perational Research，2015，247：124-136.
[12] DAHIYA S，HANDA S S，SINGH N P.A feature selection enabled hybrid-bagging algorithm for credit risk evaluation[J].Expert Systems，2017，34（9）：e12217.
[13] XIA Y，LIU C，DA B A.Novel heterogeneous ensemble credit scoring model based on stacking approach[J].Expert Systems with Applications，2018，93：182-199.
[14] LIU W，FAN H，XIA M.Multi-grained and multi-layered gradient boosting decision tree for credit scoring[J].Applied Intelligence，2021：1-17.
[15] CHEN T Q，GUESTRIN C.XGBoost：a scalable tree boosting system[C]//ACM SIGKDD 2016：2016 International Conference on Knowledge Discovery & Data Mining，2016：785-794.
[16] KALYANI P，MANASA Y，AHAMMAD S H，et al.Prediction of patient’s neurological recovery from cervical spinal cord injury through XGBoost learning approach[J].European Spine Journal，2023，32（6）：2140-2148.
[17] 王博文，王景升，吴恩重.面向不平衡数据集的SMOTENC-XGBoost驾驶人交通安全评估模型[J].科学技术与工程，2023，23（2）：831-837.
WANG B W，WANG J S，WU E Z.SMOTENC-XGBoost driver traffic safety assessment model for unbalanced dataset[J].Science Technology and Engineering，2023，23（2）：831-837.
[18] 王智勇，李丽敏，温宗周，等.基于GWO-XGBoost泥石流灾害预测[J].电子测量技术，2023，46（3）：92-99.
WANG Z Y，LI L M，WEN Z Z，et al.Debris flow disaster prediction based on GWO-XGBoos model[J].Electronic Measurement Technology，2023，46（3）：92-99.
[19] CHANG G J.Application of eXtreme gradient boosting trees in the construction of credit risk assessment models for financial institutions[J].Applied Soft Computing，2018，73：914-920.
[20] QIN C，ZHANG Y，BAO F，et al.XGBoost optimized by adaptive particle swarm optimization for credit scoring[J].Mathematical Problems in Engineering，2021：1-18.
[21] PELIKAN M.Bayesian optimization algorithm：from singl elevel to hierarchy[D].Urbana-Champaign：University of Illinois at Urbana-Champaign.Dept of Coputer Science，2002.
[22] PELIKAN M，GOLDBERG D E，CANTU-PAZ E.BOA：the Bayesan optimization algorithm[C]//Proceedings of the Genetic and Evolutionary Computation Conference（GECCO-99），1999：525-532.
[23] 江敏，陈一民.贝叶斯优化算法的发展综述[J].计算机工程与设计，2010（14）：3254-3259.
JIANG M，CHEN Y M.Survey on Bayesian optimization algorithm[J].Computer Engineering and Design，2010（14）：3254-3259.
[24] 王永生，关世杰，刘利民，等.基于XGBoost扩展金融因子的风电功率预测方法[J].浙江大学学报（工学版），2023，57（5）：1038-1049.
WANG?Y S，?GUAN?S J，LIU?L M，et al.Wind power prediction method based on XGBoost extended financial factor[J].Journal of Zhejiang University（Engineering Science），2023，57（5）：1038-1049.
[25] SNOEK J，LAROCHELLE H，ADAMS R P.Practical Bayesian optimization of machine learning algorithms[C]//Advances in Neural Information Processing Systems，2012：2951-2959.
[26] 李叶紫，王振友，周怡璐，等.基于贝叶斯最优化的Xgboost算法的改进及应用[J].广东工业大学学报，2018，35（1）：23-27.
LI Y Z，WANG Z Y，ZHOU Y L，et al.The improvement and application of Xgboost method based on the Bayesian optimization[J].Journal of Guangdong University of Technology，2018，35（1）：23-27.
[27] WU J，CHEN X Y，ZHANG H，et al.Hyperparameter optimization for machine learning models based on Bayesian optimization[J].Journal of Electronic Science and Technology，2019，17（1）：26-40.
[28] CORTES C，VAPNIK V.Support-vector networks[J].Machine Learning，1995，20：273-297.
[29] BREIMAN L.Bagging predictors[J].Machine Learning，1996，24：123-140.
[30] BREIMAN L.Random forests[J].Machine Learning，2001，45：5-32.
[31] FREUND Y，SCHAPIRE R E.A decision-theoretic generalization of on-line learning and an application to boosting[J].Journal of Computer and System Sciences，1997，55（1）：119-139.
[32] HAND D J.Classifier technology and the illusion of progress[J].Statistical Science，2006，21：1-14.
[33] FLOREZ-LOPEZ R，RAMON-JERONIMO J M.Enhancing accuracy and interpretability of ensemble strategies in credit risk assessment.A correlated-adjusted decision forest proposal[J].Expert Systems with Applications，2015，42（13）：5737-5753.