基于强化学习多算法组合模型的智能化模糊测试技术

doi:10.3778/j.issn.1002-8331.2307-0199

摘要/Abstract

摘要： 随着物联网技术的发展，物联网智能终端得到普及。当前物联网终端固件中存在大量安全漏洞，使用人工的方法对物联网终端设备进行漏洞检测存在极大不便性。目前多采用基于遗传算法的智能化模糊测试技术，使用随机变异数据对待测固件进行自动化测试。针对现有基于遗传算法的模糊测试技术存在的效率低下问题，提出了一种基于多强化学习算法组合的智能化模糊测试模型。该模型利用强化学习算法优化模糊测试变异算子选择策略，通过对不同测试用例智能化选择不同变异算子的方式提高了模糊测试代码覆盖率。通过在LAVA数据集上进行对比实验，综合比较了DDQN、DDPG、TRPO及PPO算法在模型中的表现情况，并与传统模糊测试方法进行比较，结果表明在模糊测试环境下，对于不同的目标程序，不同算法性能存在显著差异，同时基于强化学习的模糊测试方法明显优于传统模糊测试方法，证明了所提模型的可用性及有效性。

关键词: 物联网终端, 强化学习, 模糊测试, 漏洞发现

Abstract: With the development of Internet of things technology, intelligent terminals of the Internet of things have gained popularity. At present, there are many security vulnerabilities in the firmware of the Internet of things terminal, and it is very inconvenient to use manual methods to detect the vulnerabilities of the Internet of things terminal equipment. The intelligent fuzzing technology based on genetic algorithms is mainly used, and the firmware to be tested is automatically tested using random variation data. Aiming at the low efficiency of the existing fuzzing technology based on genetic algorithms, this paper proposes an intelligent fuzzing model based on multiple reinforcement learning algorithms. In this model, reinforcement learning algorithms are used to optimize the mutation operator selection strategy of fuzzing and the code coverage of fuzzing is improved by intelligently selecting different mutation operators for different test cases. This paper compares the performance of DDQN, DDPG, TRPO, and PPO algorithms in the model through comparative experiments on LAVA datasets and traditional fuzzing methods. The results show that in the fuzzing environment, there are significant differences in the performance of different algorithms for different target programs and the fuzzing method based on reinforcement learning is obviously superior to the traditional fuzzing method, proving the proposed model’s availability and effectiveness.

Key words: Internet of things terminal, reinforcement learning, fuzzing, vulnerability discovery

许爱东, 徐培明, 尚进, 孙钦东. 基于强化学习多算法组合模型的智能化模糊测试技术[J]. 计算机工程与应用, 2024, 60(20): 284-292.

XU Aidong, XU Peiming, SHANG Jin, SUN Qindong. Intelligent Fuzzing Technology Based on Combination Model of Multiple Reinforcement Learning Algorithms[J]. Computer Engineering and Applications, 2024, 60(20): 284-292.

参考文献

[1] 邹权臣, 张涛, 吴润浦, 等. 从自动化到智能化: 软件漏洞挖掘技术进展[J]. 清华大学学报 (自然科学版), 2018, 58(12): 1079-1094.
ZOU Q C, ZHANG T, WU R P, et al. From automation to intelligence: survey of research on vulnerability discovery techniques[J]. Journal of Tsinghua University (Science and Technology), 2018, 58(12): 1079-1094.
[2] MILLER B P, FREDRIKSEN L, SO B. An empirical study of the reliability of UNIX utilities[J]. Communications of the ACM, 1990, 33(12): 32-44.
[3] ZALEWSKI M. American fuzzy lop[EB/OL]. [2023-06-10]. https://lcamtuf.coredump.cx/afl/.
[4] SHAO J, ZHOU Y, LIU G, et al. Optimized mutation of grey-box fuzzing: a deep RL-based approach[C]//Proceedings of the 2023 IEEE 12th Data Driven Control and Learning Systems Conference, 2023: 1296-1300.
[5] YUN J, RUSTAMOV F, KIM J, et al. Fuzzing of embedded systems: a survey[J]. ACM Computing Surveys, 2022, 55(7): 1-33.
[6] B?HME M, PHAM V T, ROVCHOUDHURY A. Coverage-based greybox fuzzing as Markov chain[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016: 1032-1043.
[7] B?HME M, PHAM V T, NGUYEN M D, et al. Directed greybox fuzzing[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017: 2329-2344.
[8] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[9] VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2016: 2094-2100.
[10] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. arXiv:1509.
02971, 2015.
[11] CHEN D, ZHANG Y, WEI W, et al. Efficient vulnerability detection based on an optimized rule-checking static analysis technique[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(3): 332-345.
[12] BACKES M, RIECK K, SKORUPPA M, et al. Efficient and flexible discovery of PHP application vulnerabilities[C]//Proceedings of the 2017 IEEE European Symposium on Security and Privacy, 2017: 334-349.
[13] KISS B, KOSMATOV N, PARIENTE D, et al. Combining static and dynamic analyses for vulnerability detection: illustration on heartbleed[C]//Proceedings of the 11th International Haifa Verification Conference, 2015: 39-50.
[14] CHOI Y H, PARK M W, EOM J H, et al. Dynamic binary analyzer for scanning vulnerabilities with taint analysis[J]. Multimedia Tools and Applications, 2015, 74: 2301-2320.
[15] 任泽众, 郑晗, 张嘉元, 等. 模糊测试技术综述[J]. 计算机研究与发展, 2021, 58(5): 944-963.
REN Z Z, ZHENG H, ZHANG J Y, et al. A review of fuzzing techniques[J]. Journal of Computer Research and Development, 2021, 58(5): 944-963.
[16] 史记, 曾昭龙, 杨从保, 等. Fuzzing 测试技术综述[J]. 信息网络安全, 2014 (3): 87-91.
SHI J, ZENG Z L, YANG C B, et al. The summary of fuzzing testing technology[J]. Netinfo Security, 2014(3): 87-91.
[17] LI Y W, JI S L, LYU C Y, et al. V-fuzz: vulnerability prediction-assisted evolutionary fuzzing for binary programs[J]. IEEE Transactions on Cybernetics, 2020, 52(5): 3745-3756.
[18] CHEN C. Grey-box fuzzing with deep reinforcement learning and process trace back[C]//Proceedings of the 2021 4th International Conference on Advanced Electronic Materials, Computers and Software Engineering, 2021: 1167-1171.
[19] 高凤娟, 王豫, 司徒凌云, 等. 基于深度学习的混合模糊测试方法[J]. 软件学报, 2021, 32(4): 988-1005.
GAO F J, WANG Y, SITU L Y, et al. Deep learning-based hybrid fuzz testing[J]. Journal of Software, 2021, 32(4): 988-1005.
[20] B?TTINGER K, GODEFROID P, SINGH R. Deep reinforcement fuzzing[C]//Proceedings of the 2018 IEEE Security and Privacy Workshops, 2018: 116-122.
[21] DROZD W, WAGNER M D. FuzzerGym: a competitive framework for fuzzing and learning[J]. arXiv:1807.07490, 2018.
[22] libFuzzer: a library for coverage-guided fuzz testing[EB/OL]. (2016-08-31)[2023-06-10]. http://llvm.org/docs/LibFuzzer.html.
[23] ZHANG Z, CUI B J, CHEN C. Reinforcement learning-based fuzzing technology[C]//Proceedings of the 14th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, 2021: 244-253.
[24] SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv:1707.06347, 2017.
[25] SCHULMAN J, LEVINE S, ABBEEL P, et al. Trust region policy optimization[C]//Proceedings of the International Conference on Machine Learning, 2015: 1889-1897.
[26] DOLAN-GAVITT B, HULIN P, KIRDA E, et al. Lava: large-scale automated vulnerability addition[C]//Proceedings of the 2016 IEEE Symposium on Security and Privacy, 2016: 110-121.