轻梯度提升机算法的发展与应用

doi:10.3778/j.issn.1002-8331.2405-0396

摘要/Abstract

摘要： 轻梯度提升机算法（light gradient boosting machine，LightGBM）是机器学习领域中比较强大的算法之一，LightGBM采用高效的树学习算法，以更快地训练模型，其独特的直方图分桶方法和基于梯度的单边叶子生长技术降低了内存的使用和计算成本。LightGBM被广泛应用于医疗、自然语言处理、金融、工业制造等领域。然而，LightGBM在高维数据处理、类别特征处理、模型解释性等方面仍面临许多挑战。目前，解决这些问题的方法主要集中在特征工程、可视化、模型混合等方面，并取得了很好的效果。介绍了决策树家族的相关算法原理和变体研究；对LightGBM的原理、优缺点进行梳理，归纳出算法所面临的挑战，并指出LightGBM未来的研究热点和难点；对LightGBM的发展进行了总结和展望。

关键词: 轻梯度提升机算法, 决策树, 集成学习, 机器学习

Abstract: Light gradient boosting machine (LightGBM) is one of the more powerful algorithms in the field of machine learning. LightGBM uses an efficient tree learning algorithm to train models faster. Its unique histogram bucketing method and gradient-based one-sided leaf growing technique reduce memory usage and computational cost. LightGBM is widely used in medical, natural language processing, finance, industrial manufacturing and other fields. However, LightGBM still faces many challenges in high-dimensional data processing, category feature processing, and model interpretability, etc. At present, the methods to solve these problems mainly focus on feature engineering, visualization, model mixing, etc, and have achieved good results. Firstly, the algorithm principles and variants of the decision tree family are introduced. Secondly, the principles, advantages and disadvantages of LightGBM are sorted out, the challenges faced by the algorithm are summarized, and the future research hot spots and difficulties of LightGBM are pointed out. Finally, the development of LightGBM is summarized and prospected.

Key words: light gradient boosting machine (LightGBM), decision tree, ensemble learning, machine learning

魏佳妹, 袁书娟, 孔闪闪, 杨爱民, 赵晨颖. 轻梯度提升机算法的发展与应用[J]. 计算机工程与应用, 2025, 61(5): 32-42.

WEI Jiamei, YUAN Shujuan, KONG Shanshan, YANG Aimin, ZHAO Chenying. Development and Application of Light Gradient Boosting Machine[J]. Computer Engineering and Applications, 2025, 61(5): 32-42.

参考文献

[1] 罗常伟, 王双双, 尹峻松, 等. 集成学习研究现状及展望[J]. 指挥与控制学报, 2023, 9(1): 1-8.
LUO C W, WANG S S, YIN J S, et al. Research status and prospect of ensemble learning[J]. Journal of Command and Control, 2023, 9(1): 1-8.
[2] DU G S, LIU Z X, LU H F. Application of innovative risk early warning mode under big data technology in Internet credit financial risk assessment[J]. Journal of Computational and Applied Mathematics, 2021, 386: 113260.
[3] YE Y G, SHI J S, ZHU D X, et al. Management of medical and health big data based on integrated learning-based health care system: a review and comparative analysis[J]. Computer Methods and Programs in Biomedicine, 2021, 209: 106293.
[4] ZHONG X R, DUAN S K, WANG L D. An effective and efficient broad-based ensemble learning model for moderate-large scale image recognition[J]. Artificial Intelligence Review, 2023, 56(5): 4197-4215.
[5] 白少进, 白静, 司庆龙, 等. 面向三维模型多样化分类的深度集成学习[J]. 计算机工程与应用, 2023, 59(5): 222-231.
BAI S J, BAI J, SI Q L, et al. Deep ensemble learning for diversified 3D model classification[J]. Computer Engineering and Applications, 2023, 59(5): 222-231.
[6] GU Q H, SUN W J, LI X X, et al. A new ensemble classification approach based on rotation forest and lightGBM[J]. Neural Computing and Applications, 2023, 35(15): 11287-11308.
[7] MCTAVISH H, ZHONG C D, ACHERMANN R, et al. Fast sparse decision tree optimization via reference ensembles[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 9604-9613.
[8] PHAM B T, PHONG T V, NGUYEN-THOI T, et al. Ensemble modeling of landslide susceptibility using random subspace learner and different decision tree classifiers[J]. Geocarto International, 2022, 37(3): 735-757.
[9] 范劭博, 张中杰, 黄健. 决策树剪枝加强的关联规则分类方法[J]. 计算机工程与应用, 2023, 59(5): 87-94.
FAN S B, ZHANG Z J, HUANG J. Association rule classification method strengthened by decision tree pruning[J]. Computer Engineering and Applications, 2023, 59(5): 87-94.
[10] QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1): 81-106.
[11] CHANMEE S, KESORN K. Semantic decision trees: a new learning system for the ID3-based algorithm using a knowledge base[J]. Advanced Engineering Informatics, 2023, 58: 102156.
[12] KLEMEL? J, KLINKE S, SOFYAN H. Classification and regression trees[J]. Springer Nature Link, 2000: 281-304.
[13] PURWANTI E, NOR R U N U, SOELISTYONO S. Web design for stroke early detection using decision tree C5.0[J]. Komputasi: Jurnal Ilmiah Ilmu Komputer Dan Matematika, 2023, 20(2): 135-147.
[14] KUSHIRO S, FUKUI S, INUI A, et al. Clinical prediction rule for bacterial arthritis: Chi-squared automatic interaction detector decision tree analysis model[J]. SAGE Open Medicine, 2023, 11(3): 1-10.
[15] ZAITSEVA E, RABCAN J, LEVASHENKO V, et al. Importance analysis of decision making factors based on fuzzy decision trees[J]. Applied Soft Computing, 2023, 134: 109988.
[16] HILLEBRAND E, LUKAS M, WEI W. Bagging weak predictors[J]. International Journal of Forecasting, 2021, 37(1): 237-254.
[17] GENUER R, POGGI J M, GENUER R, et al. Random forests[M]. Berlin: Springer International Publishing, 2020.
[18] CHAVENT M, GENUER R, SARACCO J. Combining clustering of variables and feature selection using random forests[J]. Communications in Statistics-Simulation and Computation, 2021, 50(2): 426-445.
[19] WAN Y, LIU D, REN J C. A modeling method of wide random forest multi-output soft sensor with attention mechanism for quality prediction of complex industrial processes[J]. Advanced Engineering Informatics, 2024, 59: 102255.
[20] LI Z, VAN LEEUWEN M. Explainable contextual anomaly detection using quantile regression forests[J]. Data Mining and Knowledge Discovery, 2023, 37(6): 2517-2563.
[21] XU H Z, PANG G S, WANG Y J, et al. Deep isolation forest for anomaly detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(12): 12591-12604.
[22] SIGRIST F. Gaussian process boosting[J]. The Journal of Machine Learning Research, 2022, 23(1): 10565-10610.
[23] SAHIN E K. Comparative analysis of gradient boosting algorithms for landslide susceptibility mapping[J]. Geocarto International, 2022, 37(9): 2441-2465.
[24] CHEN S Q, JIANG W, CHEN J M, et al. Research on abnormal electrical behavior identification technology based on cost sensitive RAE-AdaBoost algorithm[C]//Proceedings of the 2023 IEEE 6th International Electrical and Energy Conference. Piscataway: IEEE, 2023: 2609-2614.
[25] MITSUBOSHI R, HATANO K, TAKIMOTO E. Solving linear regression with insensitive loss by boosting[J]. IEICE Transactions on Information and Systems, 2024, 107(3): 294-300.
[26] TIAN N, YANG J, WANG J J, et al. Integrated GBDT and logistic power customer complaint early warning method[C]//Proceedings of the 7th International Symposium on Advances in Electrical, Electronics, and Computer Engineering, 2022: 1478-1483.
[27] IOSIPOI L, VAKHRUSHEV A. SketchBoost: fast gradient boosted decision tree for multioutput problems[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, 2022: 25422-25435.
[28] BERKANI S, GRYECH I, GHOGHO M, et al. Data driven forecasting models for urban air pollution: moreair case study[J]. IEEE Access, 2023, 11: 133131-133142.
[29] EMAMI S, MARTíNEZ-MU?OZ G. Deep learning for multi-output regression using gradient boosting[J]. IEEE Access, 2024, 12: 17760-17772.
[30] LIU Z K, JIANG P, DE BOCK K W, et al. Extreme gradient boosting trees with efficient Bayesian optimization for profit-driven customer churn prediction[J]. Technological Forecasting and Social Change, 2024, 198: 122945.
[31] KAVZOGLU T, TEKE A. Predictive performances of ensemble machine learning algorithms in landslide susceptibility mapping using random forest, extreme gradient boosting (XGBoost) and natural gradient boosting (NGBoost)[J]. Arabian Journal for Science and Engineering, 2022, 47(6): 7367-7385.
[32] ARIF ALI Z, ABDULJABBAR Z H, TAHIR H A, et al. eXtreme gradient boosting algorithm with machine learning: a review[J]. Academic Journal of Nawroz University, 2023, 12(2): 320-334.
[33] PUNURI S B, KUANAR S K, KOLHAR M, et al. Efficient net-XGBoost: an implementation for facial emotion recognition using transfer learning[J]. Mathematics, 2023, 11(3): 776.
[34] SHUA Q Q, PENG H B, LI J K. Landslide susceptibility prediction modelling based on semi-supervised XGBoost model[J]. Geological Journal, 2024, 59(9): 2655-2667.
[35] MENG Q. LightGBM: a highly efficient gradient boosting decision tree[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, 2017: 3146-3154.
[36] REN J, YU Z P, GAO G L, et al. A CNN-LSTM-LightGBM based short-term wind power prediction method based on attention mechanism[J]. Energy Reports, 2022, 8: 437-443.
[37] BRESCI V, LEMOINE M, GREMILLET L, et al. Nonresonant particle acceleration in strong turbulence: comparison to kinetic and MHD simulations[J]. Physical Review D, 2022, 106(2): 023028.
[38] XU K, HAN Z T, XU H S, et al. Rapid prediction model for urban floods based on a light gradient boosting machine approach and hydrological-hydraulic model[J]. International Journal of Disaster Risk Science, 2023, 14(1): 79-97.
[39] HARTANTO A D, NUR KHOLIK Y, PRISTYANTO Y. Stock price time series data forecasting using the light gradient boosting machine (LightGBM) model[J]. International Journal on Informatics Visualization, 2023, 7(4): 2270.
[40] ZHANG X. A model combining LightGBM and neural network for high-frequency realized volatility forecasting[C]//Proceedings of the Advances in Economics, Business and Management Research, 2022.
[41] DEMIR S, SAHIN E K. Predicting occurrence of liquefaction-induced lateral spreading using gradient boosting algorithms integrated with particle swarm optimization: pso-XGBoost, PSO-LightGBM, and PSO-CatBoost[J]. Acta Geotechnica, 2023, 18(6): 3403-3419.
[42] ZHANG T L, LI S J, JING X R, et al. Negative comment recognition model based on LightGBM[C]//Proceedings of the Second International Conference on Statistics, Applied Mathematics, and Computing Science, 2023: 179.
[43] REHMAN A, NAZ S, RAZZAK I. Leveraging big data analytics in healthcare enhancement: trends, challenges and opportunities[J]. Multimedia Systems, 2022, 28(4): 1339-1371.
[44] 杨顺, 郝晓燕, 马垚, 等. 基于生成对抗网络的差分隐私生成数据方法[J]. 计算机工程与设计, 2024, 45(1): 39-46.
YANG S, HAO X Y, MA Y, et al. Differential privacy generation data method based on generative adversarial network[J]. Computer Engineering and Design, 2024, 45(1): 39-46.
[45] ZHAO B, LI B, ZHANG J Q, et al. DCLGM: fusion recommendation model based on LightGBM and deep learning[J]. Neural Processing Letters, 2024, 56(1): 17.
[46] CAO Q, WU Y H, YANG J, et al. Greenhouse temperature prediction based on time-series features and LightGBM[J]. Applied Sciences, 2023, 13(3): 1610.
[47] LYU Z Q. A novel LightGBM-based industrial Internet intrusion detection method[J]. International Journal of Computer Applications in Technology, 2023, 71(3): 208-216.
[48] LIM Z Y, PANG Y H, OOI S Y, et al. Analysis of an optimized LightGBM based on different objective functions for customer churn prediction in telecom[C]//Proceedings of the 2023 7th International Conference on Automation, Control and Robots. Piscataway: IEEE, 2023: 102-107.
[49] BHUTTA A A, NISA M U, MIAN A N. Lightweight real-time WiFi-based intrusion detection system using LightGBM[J]. Wireless Networks, 2024, 30(2): 749-761.
[50] RUFO D D, DEBELEE T G, IBENTHAL A, et al. Diagnosis of diabetes mellitus using gradient boosting machine (LightGBM)[J]. Diagnostics, 2021, 11(9): 1714.
[51] HAMED E A, SALEM M A, BADR N L, et al. An efficient combination of convolutional neural network and LightGBM algorithm for lung cancer histopathology classification[J]. Diagnostics, 2023, 13(15): 2469.
[52] GAYATHRI S, ANAND A, VIJAYVARGIYA A, et al. EmoSens: emotion Recognition based on Sensor data analysis using LightGBM[C]//Proceedings of the 2022 IEEE International Conference on Electronics, Computing and Communication Technologies. Piscataway: IEEE, 2022: 1-6.
[53] REGO R C B, SILVA V M, FERNANDES V M. Predicting gender by first name using character-level machine learning[J]. arXiv:2106.10156, 2021.
[54] WANG D N, LI L, ZHAO D. Corporate finance risk prediction based on LightGBM[J]. Information Sciences, 2022, 602: 259-268.
[55] TANG M Z, MENG C H, WU H W, et al. Fault detection for wind turbine blade bolts based on GSG combined with CS-LightGBM[J]. Sensors, 2022, 22(18): 6763.
[56] SANI S H, XIA H B, MILISAVLJEVIC-SYED J, et al. Supply chain 4.0: a machine learning-based Bayesian-optimized LightGBM model for predicting supply chain risk[J]. Machines, 2023, 11(9): 888.
[57] GHIASI M M, ZENDEHBOUDI S. Application of decision tree-based ensemble learning in the classification of breast cancer[J]. Computers in Biology and Medicine, 2021, 128: 104089.
[58] HANCOCK J, KHOSHGOFTAAR T M. Leveraging LightGBM for categorical big data[C]//Proceedings of the 2021 IEEE Seventh International Conference on Big Data Computing Service and Applications. Piscataway: IEEE, 2021: 149-154.
[59] 于涛, 丁海旭, 黄卫民, 等. 面向复杂异质数据的集成学习研究综述[J]. 控制工程, 2023, 30(8): 1425-1435.
YU T, DING H X, HUANG W M, et al. A survey of ensemble learning for complex heterogeneous data[J]. Control Engineering of China, 2023, 30(8): 1425-1435.
[60] 邢长征, 徐佳玉. LightGBM混合模型在乳腺癌诊断中的应用[J]. 计算机工程与应用, 2024, 60(6): 330-338.
XING C Z, XU J Y. Hybrid LightGBM model for breast cancer diagnosis[J]. Computer Engineering and Applications, 2024, 60(6): 330-338.