基于分层和随机策略的鲁棒随机森林

doi:10.3778/j.issn.1002-8331.2407-0405

摘要/Abstract

摘要： 近年来，可解释的树模型在对抗性攻击下的鲁棒性受到极大关注。尽管已经取得了显著的进展，但仍存在着一些问题。树模型的鲁棒训练算法虽然有效提升了模型鲁棒性，但同时伴随着训练成本的增加以及自然准确率的下降，呈现出与对抗性训练相似的局限性。为了解决这一挑战，针对随机森林模型进行实验和分析，提出一种基于分层和随机策略的鲁棒随机森林算法，该算法使用高斯衰减函数根据树的深度分层衰减分割算法中引入的扰动值，逐层递减扰动带来的负面影响，并引入基于伯努利分布的随机策略控制森林中树的划分过程，以协调模型在自然场景和对抗性攻击下的性能。旨在找到一种平衡策略，既能提升模型在面对对抗性攻击时的鲁棒性，又维持其在标准数据集上的高性能表现。在真实世界数据集上的大量实验表明，提出的算法能够在确保模型具备一定的鲁棒性的同时，显著减轻由鲁棒训练导致的准确率损失，提高模型的泛化能力。

关键词: 随机森林, 对抗训练, 鲁棒性, 机器学习

Abstract: In recent years, the robustness of explainable tree models against adversarial attacks has garnered significant attention. Despite notable progress, several challenges remain. Robust training algorithms for tree models, although effective in enhancing model robustness, also lead to increased training costs and decreased natural accuracy, reflecting the similar limitation as adversarial training. To address this challenge, experiments and analyses are conducted on random forest models, resulting in the proposal of a robust random forest algorithm that is grounded in a hierarchical and random strategy. This algorithm employs a Gaussian attenuation function to hierarchically attenuate the perturbation values introduced in the splitting algorithm according to the depth of the tree, thereby progressively reducing the negative impact of perturbations. Additionally, a random strategy based on Bernoulli distribution is introduced to control the splitting process of trees in the forest, aiming to balance the model’s performance under both natural scenarios and adversarial attacks. The goal is to find a strategy that enhances the model’s robustness against adversarial attacks while maintaining high performance on standard datasets. Extensive experiments on real-world datasets demonstrate that the proposed algorithm can ensure a certain level of robustness while significantly mitigating the accuracy loss caused by robust training, thus improving the model’s generalization ability.

Key words: random forest, adversarial training, robustness, machine learning

刘昭仪, 王士同. 基于分层和随机策略的鲁棒随机森林[J]. 计算机工程与应用, 2025, 61(21): 203-213.

LIU Zhaoyi, WANG Shitong. Robust Random Forests Based on Hierarchical and Random Strategy[J]. Computer Engineering and Applications, 2025, 61(21): 203-213.

参考文献

[1] 赵延玉, 赵晓永, 王磊, 等. 可解释人工智能研究综述[J]. 计算机工程与应用, 2023, 59(14): 1-14.
ZHAO Y Y, ZHAO X Y, WANG L, et al. Review of explainable artificial intelligence[J]. Computer Engineering and Applications, 2023, 59(14): 1-14.
[2] TJOA E, GUAN C T. A survey on explainable artificial intelligence (XAI): toward medical XAI[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(11): 4793-4813.
[3] BREIMAN L. Random forests[J]. Machine Learning, 2001, 45(1): 5-32.
[4] ZHOU Z H, FENG J. Deep forest[J]. National Science Review, 2019, 6(1): 74-86.
[5] 徐杨宇, 高宝元, 郭杰龙, 等. 尺度不变的条件数约束的模型鲁棒性增强算法[J]. 计算机工程与应用, 2024, 60(8): 140-147.
XU Y Y, GAO B Y, GUO J L, et al. Model robustness enhancement algorithm with scale invariant condition number constraint[J]. Computer Engineering and Applications, 2024, 60(8): 140-147.
[6] COHEN J M, ROSENFELD E, KOLTER J Z. Certified adversarial robustness via randomized smoothing[J]. arXiv:1902.02918, 2019.
[7] MADRY A, MAKELOV A, SCHMIDT L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv:1706.06083, 2017.
[8] SALMAN H, LI J, RAZENSHTEYN I, et al. Provably robust deep learning via adversarially trained smoothed classifiers[C]//Advances in Neural Information Processing Systems, 2019: 11289-11300.
[9] ZHANG H Y, YU Y D, JIAO J T, et al. Theoretically principled trade-off between robustness and accuracy[J]. arXiv:1901.08573, 2019.
[10] RAGHUNATHAN A, XIE S M, YANG F, et al. Adversarial training can hurt generalization[J]. arXiv:1906.06032, 2019.
[11] YANG Y Y, RASHTCHIAN C, ZHANG H, et al. A closer look at accuracy vs. robustness[C]//Advances in Neural Information Processing Systems, 2020: 8588-8601.
[12] KANTCHELIAN A, TYGAR J D, JOSEPH A D. Evasion and hardening of tree ensemble classifiers[J]. arXiv:1509. 07892, 2015.
[13] CHEN H G, ZHANG H, BONING D, et al. Robust decision trees against adversarial examples[J]. arXiv:1902.10660, 2019.
[14] ANDRIUSHCHENKO M, HEIN M. Provably robust boosted decision stumps and trees against adversarial attacks[C]//Proceedings of the Neural Information Processing Systems, 2019.
[15] VOS D, VERWER S. Robust optimal classification trees against adversarial examples[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 8520-8528.
[16] CHEN Y Z, WANG S Q, JIANG W F, et al. Cost-aware robust tree ensembles for security applications[J]. arXiv:1912.01149, 2019.
[17] GUO J Q, TENG M Z, GAO W, et al. Fast provably robust decision trees and boosting[C]//Proceedings of the 39th International Conference on Machine Learning, 2022: 8127-8144.
[18] RANZATO F, ZANELLA M. Genetic adversarial training of decision trees[C]//Proceedings of the Genetic and Evolutionary Computation Conference. New York: ACM, 2021: 358-367.
[19] ?YCHOWSKI A, PERRAULT A, MA?DZIUK J. Coevolutionary algorithm for building robust decision trees under minimax regret[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2024: 21869-21877.
[20] BIGGIO B, CORONA I, MAIORCA D, et al. Evasion attacks against machine learning at test time[C]//Proceedings of the Machine Learning and Knowledge Discovery in Databases. Berlin: Springer, 2013: 387-402.
[21] QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1): 81-106.
[22] BREIMAN L. Classification and regression trees[M]. New York: Chapman and Hall, 2017.
[23] 张锟滨, 陈玉明, 吴克寿, 等. 粒向量驱动的随机森林分类算法研究[J]. 计算机工程与应用, 2024, 60(3): 148-156.
ZHANG K B, CHEN Y M, WU K S, et al. Research on granule vectors random forest classification algorithm[J]. Computer Engineering and Applications, 2024, 60(3): 148-156.
[24] DIETTERICH T G. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization[J]. Machine Learning, 2000, 40(2): 139-157.
[25] CHENG M H, LE T, CHEN P Y, et al. Query-efficient hard-label black-box attack: an optimization-based approach[J]. arXiv:1807.04457, 2018.