无人驾驶深度强化学习决策模型性能评测方法综述

doi:10.3778/j.issn.1002-8331.2502-0072

摘要/Abstract

摘要： 目前，以深度强化学习（deep reinforcement learning，DRL）为主要决策方法的端到端无人驾驶技术在典型交通驾驶任务中的表现取得显著进展。但是，由于DRL“试错”交互的独特学习方式，使其在应用到真实驾驶环境之前，必须经过严格的多维评测过程。因此，性能评测成为DRL无人驾驶决策模型向真实世界迁移的一个关键且不可或缺的步骤。梳理分析当前无人驾驶领域主流的技术实现方法；聚焦DRL方法，综述其在无人驾驶决策中的研究模式与最新成果，探讨其在处理无人驾驶任务时所面临的问题与瓶颈；面向端到端DRL无人驾驶决策模型，从安全性、鲁棒性、舒适性、效率、可靠性五个方面全面综述性能评测方法，分析影响因素并梳理性能评测流程；对比总结目前常用且开源的无人驾驶虚拟仿真平台的特点及适用场景；概述性能评测存在的开放性问题及对未来评测方法的研究展望，为相关研究和模型应用部署提供理论支持和参考依据。

关键词: 智能交通, 无人驾驶, 深度强化学习, 评测方法, 决策性能, 端到端控制

Abstract: Recently, end-to-end autonomous driving technology, using deep reinforcement learning (DRL) as the primary decision-making method, has achieved significant progress in complex dynamic environments. However, due to the unique trial-and-error learning mechanism of DRL, it must undergo a strict multi-dimensional evaluation process before being applied to real-world driving environments. Thus, performance evaluation becomes a critical and indispensable step for transferring DRL-based autonomous driving decision models to the real world. Firstly, this paper reviews the mainstream technical implementation methods in the field of autonomous driving. Next, the focus is placed on DRL methods, summarizing their research paradigms and the latest advances in autonomous driving decision-making, and discussing the issues and bottlenecks they face when addressing autonomous driving tasks. Following this, a comprehensive review of performance evaluation methods for end-to-end DRL autonomous driving decision models is provided, covering aspects such as safety, robustness, comfort, efficiency, and reliability, analyzing influencing factors, and summarizing the performance evaluation process. Subsequently, commonly used and open-source virtual simulation platforms for autonomous driving are compared and summarized regarding their features and applicable scenarios. Finally, open issues in performance evaluation and prospects for future research on evaluation methods are presented, providing theoretical support and reference for related research and model deployment.

Key words: smart transportation, autonomous driving, deep reinforcement learning, evaluation methods, decision performance, end-to-end control

顾同成, 徐东伟, 孙成巨. 无人驾驶深度强化学习决策模型性能评测方法综述[J]. 计算机工程与应用, 2025, 61(19): 12-42.

GU Tongcheng, XU Dongwei, SUN Chengju. Review of Performance Evaluation Methods for Deep Reinforcement Learning Decision Models in Autonomous Driving[J]. Computer Engineering and Applications, 2025, 61(19): 12-42.

参考文献

[1] CHEN L, WU P H, CHITTA K, et al. End-to-end autonomous driving: challenges and frontiers[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12): 10164-10183.
[2] DUAN Y, ZHANG Q, XU R. Prompting multi-modal tokens to enhance end-to-end autonomous driving imitation learning with LLMs[J]. arXiv:2404.04869, 2024.
[3] KIRAN B R, SOBH I, TALPAERT V, et al. Deep reinforcement learning for autonomous driving: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(6): 4909-4926.
[4] KARKUS P, IVANOVIC B, MANNOR S, et al. Diffstack: a differentiable and modular control stack for autonomous vehicles[C]//Proceedings of the Conference on Robot Learning, 2023: 2170-2180.
[5] HARIS M, GLOWACZ A. Navigating an automated driving vehicle via the early fusion of multi-modality[J]. Sensors, 2022, 22(4): 1425.
[6] HUANG Z Y, WU J D, LV C. Efficient deep reinforcement learning with imitative expert priors for autonomous driving[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(10): 7391-7403.
[7] HU A, CORRADO G, GRIFFITHS N, et al. Model-based imitation learning for urban driving[C]//Advances in Neural Information Processing Systems, 2022: 20703-20716.
[8] PANIEGO S, CALVO-PALOMINO R, CA?AS J. Enhancing end-to-end control in autonomous driving through kinematic-infused and visual memory imitation learning[J]. Neurocomputing, 2024, 600: 128161.
[9] AL OZAIBI Y, HINA M D, RAMDANE-CHERIF A. End-to-end autonomous driving in CARLA: a survey[J]. IEEE Access, 2024, 12: 146866-146900.
[10] PARK J B, LEE J, BACK M, et al. End-to-end driving via self-supervised imitation learning using camera and lidar data[J]. arXiv:2308.14329, 2023.
[11] ANZALONE L, BARRA P, BARRA S, et al. An end-to-end curriculum learning approach for autonomous driving scenarios[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(10): 19817-19826.
[12] ACERBO F S, SWEVERS J, TUYTELAARS T, et al. Evaluation of MPC-based imitation learning for human-like autonomous driving[J]. IFAC-PapersOnLine, 2023, 56(2): 4871-4876.
[13] LIANG H B, DONG Z B, MA Y, et al. A hierarchical imitation learning-based decision framework for autonomous driving[C]//Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. New York: ACM, 2023: 4695-4701.
[14] CULTRERA L, BECATTINI F, SEIDENARI L, et al. Addressing limitations of state-aware imitation learning for autonomous driving[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 2946-2955.
[15] CUI J P, YUAN L, XIAO W D, et al. SEAE: stable end-to-end autonomous driving using event-triggered attention and exploration-driven deep reinforcement learning[J]. Displays, 2025, 87: 102946.
[16] LI Z Y, SHANG T Y, XU P J. Multi-modal attention perception for intelligent vehicle navigation using deep reinforcement learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, 26(6): 8657-8669.
[17] LI G F, YANG Y F, LI S, et al. Decision making of autonomous vehicles in lane change scenarios: deep reinforcement learning approaches with risk awareness[J]. Transportation Research Part C: Emerging Technologies, 2022, 134: 103452.
[18] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[19] VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2016.
[20] SCHAUL T, QUAN J, ANTONOGLOU I, et al. Prioritized experience replay[C]//Proceedings of the International Conference on Learning Representations, 2015.
[21] WANG Z, SCHAUL T, HESSEL M, et al. Dueling network architectures for deep reinforcement learning[C]//Proceedings of the 33rd International Conference on Machine Learning, 2016.
[22] HAUSKNECHT M J, STONE P. Deep recurrent Q-learning for partially observable MDPs[C]//Proceedings of the 2015 AAAI Fall Symposium Series, 2015.
[23] MNIH V, BADIA A P, MIRZA M, et al. Asynchronous methods for deep reinforcement learning[C]//Proceedings of the International Conference on Machine Learning, 2016.
[24] SCHULMAN J, LEVINE S, ABBEEL P, et al. Trust region policy optimization[C]//Proceedings of the International Conference on Machine Learning, 2015.
[25] SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv:1707.06347, 2017.
[26] HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proceedings of the 35th International Conference on Machine Learning, 2018.
[27] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[C]//Proceedings of the International Conference on Learning Representations, 2016.
[28] HORGAN D, QUAN J, BUDDEN D, et al. Distributed prioritized experience replay[C]//Proceedings of the 6th International Conference on Learning Representations, 2018.
[29] ESPEHOLT L, SOYER H, MUNOS R, et al. IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures[C]//Proceedings of the 35th International Conference on Machine Learning, 2018.
[30] HESSEL M, MODAYIL J, VAN HASSELT H, et al. Rainbow: combining improvements in deep reinforcement learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[31] ZHU Z Y, ZHAO H J. A survey of deep RL and IL for autonomous driving policy learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 14043-14065.
[32] 褚端峰, 王如康, 王竞一, 等. 端到端自动驾驶的研究进展及挑战[J]. 中国公路学报, 2024, 37(10): 209-232.
CHU D F, WANG R K, WANG J Y, et al. End-to-end autonomous driving: advancements and challenges[J]. China Journal of Highway and Transport, 2024, 37(10): 209-232.
[33] CAI P D, WANG H L, SUN Y X, et al. DQ-GAT: towards safe and efficient autonomous driving with deep Q-learning and graph attention networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 21102-21112.
[34] DENG Z Y, SHI Y J, SHEN W M. V2X-lead: lidar-based end-to-end autonomous driving with vehicle-to-everything communication integration[C]//Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2023: 7471-7478.
[35] JIN Y L, JI Z Y, ZENG D, et al. VWP: an efficient DRL-based autonomous driving model[J]. IEEE Transactions on Multimedia, 2022, 26: 2096-2108.
[36] LIU J, YIN J W, JIANG Z M, et al. Attention-based distributional reinforcement learning for safe and efficient autonomous driving[J]. IEEE Robotics and Automation Letters, 2024, 9(9): 7477-7484.
[37] NAN S, NGUYEN T A, CHOI E, et al. SHANGUS: deep reinforcement learning meets heuristic optimization for speedy frontier-based exploration of autonomous vehicles in unknown spaces[J]. arXiv:2407.18892, 2024.
[38] XIAO W X, YANG Y Y, MU X Y, et al. Decision-making for autonomous vehicles in random task scenarios at unsignalized intersection using deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2024, 73(6): 7812-7825.
[39] RAMESH G, BUDATI A K, ISLAM S, et al. Artificial intelligence enabled future wireless electric vehicles with multi-model learning and decision making models[J]. Tsinghua Science and Technology, 2024, 29(6): 1776-1784.
[40] LIN K, LI Y J, CHEN S Y, et al. Motion planner with fixed-horizon constrained reinforcement learning for complex autonomous driving scenarios[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 1577-1588.
[41] BEN NAVEED K, QIAO Z Q, DOLAN J M. Trajectory planning for autonomous vehicles using hierarchical reinforcement learning[C]//Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference. Piscataway: IEEE, 2021: 601-606.
[42] XIA Y Y, LIU S C, YU Q L, et al. Parameterized decision-making with multi-modality perception for autonomous driving[C]//Proceedings of the 2024 IEEE 40th International Conference on Data Engineering. Piscataway: IEEE, 2024: 4463-4476.
[43] QIAO Z Q, TYREE Z, MUDALIGE P, et al. Hierarchical reinforcement learning method for autonomous vehicle behavior planning[C]//Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2020: 6084-6089.
[44] KHALIL Y H, MOUFTAH H T. Exploiting multi-modal fusion for urban autonomous driving using latent deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2023, 72(3): 2921-2935.
[45] DOSOVITSKIY A, ROS G, CODEVILLA F, et al. Carla: an open urban driving simulator[C]//Proceedings of the Conference on Robot Learning, 2017: 1-16.
[46] LEE S H, JUNG Y, SEO S W. Imagination-augmented hierarchical reinforcement learning for safe and interactive autonomous driving in urban environments[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(12): 19522-19535.
[47] XU D W, LIU P W, LI H J, et al. Multi-view graph convolution network reinforcement learning for CAVs cooperative control in highway mixed traffic[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 2588-2599.
[48] XU D W, ZHANG B, QIU Q W, et al. Graph-based multi agent reinforcement learning for on-ramp merging in mixed traffic[J]. Applied Intelligence, 2024, 54(8): 6400-6414.
[49] JIA X, PENG J, LIU Y J, et al. Car-following safe headway strategy with battery-health conscious: a reinforcement learning approach[C]//Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics. Piscataway: IEEE, 2020: 2432-2437.
[50] YU C, WANG X, XU X, et al. Distributed multiagent coordinated learning for autonomous driving in highways based on dynamic coordination graphs[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(2): 735-748.
[51] ROHITH G, DEVIKA K B, MENON P P, et al. An adaptive time-headway policy for lower energy consumption in autonomous vehicle platoons[C]//Proceedings of the 2022 European Control Conference. Piscataway: IEEE, 2022: 1734-1739.
[52] EL-HANSALI Y, FARRAG S, YASAR A, et al. Using surrogate measures to evaluate the safety of autonomous vehicles[J]. Procedia Computer Science, 2021, 191: 151-159.
[53] ALGHODHAIFI H, LAKSHMANAN S. Safety model of automated vehicle-VRU conflict under uncertain weather conditions and sensors failure[C]//Proceedings of SPIE: Unmanned Systems Technology XXII, 2020: 56-65.
[54] REMPE D, PHILION J, GUIBAS L J, et al. Generating useful accident-prone driving scenarios via a learned traffic prior[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 17305-17315.
[55] FENG S, YAN X T, SUN H W, et al. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment[J]. Nature Communications, 2021, 12(1): 748.
[56] FENG S, SUN H W, YAN X T, et al. Dense reinforcement learning for safety validation of autonomous vehicles[J]. Nature, 2023, 615(7953): 620-627.
[57] JIANG Z M, PAN W B, LIU J, et al. Efficient and unbiased safety test for autonomous driving systems[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(5): 3336-3348.
[58] 赵祥模, 赵玉钰, 景首才, 等. 面向自动驾驶测试的危险变道场景泛化生成[J]. 自动化学报, 2023, 49(10): 2211-2223.
ZHAO X M, ZHAO Y Y, JING S C, et al. Generalization generation of hazardous lane-changing scenarios for automated vehicle testing[J]. Acta Automatica Sinica, 2023, 49(10): 2211-2223.
[59] CHANG C, WANG S Q, ZHANG J W, et al. LLMScenario: large language model driven scenario generation[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024, 54(11): 6581-6594.
[60] ZHANG J W, XU C J, LI B. ChatScene: knowledge-enabled safety-critical scenario generation for autonomous vehicles[C]//Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2024: 15459-15469.
[61] MEI Y, NIE T, SUN J, et al. LLM-attacker: enhancing closed-loop adversarial scenario generation for autonomous driving with large language models[J]. arXiv:2501.15850, 2025.
[62] AHMED M, ABOBAKR A, LIM C P, et al. Policy-based reinforcement learning for training autonomous driving agents in urban areas with affordance learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(8): 12562-12571.
[63] CHOWDHURY J, SHIVARAMAN V, SUNDARAM S, et al. Graph-based prediction and planning policy network (GP3Net) for scalable self-driving in dynamic environments using deep reinforcement learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2024: 11606-11614.
[64] NEHME G, DEO T Y. Safe navigation: training autonomous vehicles using deep reinforcement learning in CARLA[J]. arXiv:2311.10735, 2023.
[65] HA T, OH J, LEE G M, et al. RIANet++: road graph and image attention networks for robust urban autonomous driving under road changes[J]. IEEE Robotics and Automation Letters, 2023, 8(11): 7815-7822.
[66] PAN J X, ZHOU C Y, GLADKOVA M, et al. Robust autonomous vehicle pursuit without expert steering labels[J]. IEEE Robotics and Automation Letters, 2023, 8(10): 6595-6602.
[67] CHITTA K, PRAKASH A, JAEGER B, et al. TransFuser: imitation with transformer-based sensor fusion for autonomous driving[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 12878-12895.
[68] 苏珂, 潘浩然. 无人驾驶环境下行人过路意图识别技术研究综述[J/OL]. 计算机集成制造系统, 2024: 1-30 (2024-11-18)[2025-04-01]. https://link.cnki.net/doi/10.13196/j.cims.2024.
0437.
SU K, PAN H R. Review on the identification technology of pedestrian crossing intention in unmanned driving environment[J/OL]. Computer Integrated Manufacturing Systems, 2024: 1-30 (2024-11-18)[2025-04-01]. https://link.cnki.net/doi/10.13196/j.cims.2024.0437.
[69] MEDINA-LEE J F, JIMéNEZ V, GODOY J, et al. Maneuver planner for automated vehicles on urban scenarios[C]//Proceedings of the 2022 IEEE International Conference on Vehicular Electronics and Safety. Piscataway: IEEE, 2022: 1-7.
[70] RETTENMAIER M, DINKEL S, BENGLER K. Communication via motion-suitability of automated vehicle movements to negotiate the right of way in road bottleneck scenarios[J]. Applied Ergonomics, 2021, 95: 103438.
[71] XING Y, ZHAO C, LI Z H, et al. A right-of-way based strategy to implement safe and efficient driving at non-signalized intersections for automated vehicles[J]. arXiv:1905.01150, 2019.
[72] LI L, ZHAO C, WANG X, et al. Three principles to determine the right-of-way for AVs: safe interaction with humans[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 7759-7774.
[73] 张新钰, 卢毅果, 高鑫, 等. 面向智能网联汽车的车路协同感知技术及发展趋势[J]. 自动化学报, 2025, 51(2): 233-248.
ZHANG X Y, LU Y G, GAO X, et al. Vehicle-road collaborative perception technology and development trend for intelligent connected vehicles[J]. Acta Automatica Sinica, 2025, 51(2): 233-248.
[74] WANG J, SUN H Y, ZHU C. Vision-based autonomous driving: a hierarchical reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2023, 72(9): 11213-11226.
[75] CHOWDHURY J, VEERENDRANATH V, SUNDARAM S, et al. Predictive maneuver planning with deep reinforcement learning (PMP-DRL) for comfortable and safe autonomous driving[J]. arXiv:2306.09055, 2023.
[76] ALEXIADIS V, COLYAR J, HALKIAS J, et al. The next generation simulation program[J]. ITE Journal, 2004, 74(8): 22.
[77] COLYAR J, HALKIAS J. US highway 101 dataset: FHWA-HRT-07-030[R]. Federal Highway Administration Research and Technology, 2007.
[78] HALKIAS J, COLYAR J. Interstate 80 freeway dataset: FHWA-HRT-06-137[J]. Federal Highway Administration Research and Technology, 2006.
[79] BEN ELALLID B, EL ALAOUI H, BENAMAR N. Deep reinforcement learning for autonomous vehicle intersection navigation[C]//Proceedings of the 2023 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies. Piscataway: IEEE, 2023: 308-313.
[80] 马晨, 沈超, 蔺琛皓, 等. 针对自动驾驶智能模型的攻击与防御[J]. 计算机学报, 2024, 47(6): 1431-1452.
MA C, SHEN C, LIN C H, et al. Attacks and defenses for autonomous driving intelligence models[J]. Chinese Journal of Computers, 2024, 47(6): 1431-1452.
[81] 陈博言, 沈晴霓, 张晓磊, 等. 智能网联汽车的车载网络攻防技术研究进展[J]. 软件学报, 2025, 36(1): 341-370.
CHEN B Y, SHEN Q N, ZHANG X L, et al. Research progress on attacks and defenses technologies for in-vehicle network of intelligent connected vehicle[J]. Journal of Software, 2025, 36(1): 341-370.
[82] XIE Z Q, XIANG Y X, LI Y K, et al. Security analysis of poisoning attacks against multi-agent reinforcement learning[C]//Proceedings of the International Conference on Algorithms and Architectures for Parallel Processing. Cham: Springer, 2022: 660-675.
[83] ZHANG X, MA Y, SINGLA A, et al. Adaptive reward-poisoning attacks against reinforcement learning[C]//Proceedings of the International Conference on Machine Learning, 2020: 11225-11234.
[84] AN Y, YANG W, CHOI D. Locality-based action-poisoning attack against the continuous control of an autonomous driving model[J]. Processes, 2024, 12(2): 314.
[85] LIU G, LAI L. Provably efficient black-box action poisoning attacks against reinforcement learning[C]//Advances in Neural Information Processing Systems, 2021: 12400-12410.
[86] XU H. Environment poisoning in reinforcement learning: attacks and resilience[D]. Singapore: Nanyang Technological University, 2023.
[87] CHEN J Y, WANG X K, ZHANG Y, et al. Agent manipulator: stealthy strategy attacks on deep reinforcement learning[J]. Applied Intelligence, 2023, 53(10): 12831-12858.
[88] WANG Y, SARKAR E, JABARI S E, et al. On the vulnerability of deep reinforcement learning to backdoor attacks in autonomous vehicles[M]//Embedded machine learning for cyber-physical, IoT, and edge computing: use cases and emerging challenges. Cham: Springer, 2023: 315-341.
[89] WANG Y, SARKAR E, LI W Q, et al. Stop-and-go: exploring backdoor attacks on deep reinforcement learning-based traffic congestion control systems[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 4772-4787.
[90] YU Y B, LIU J J, LI S Q, et al. A temporal-pattern backdoor attack to deep reinforcement learning[C]//Proceedings of the GLOBECOM 2022-2022 IEEE Global Communications Conference. Piscataway: IEEE, 2022: 2710-2715.
[91] HOSSEINI A, HOUTI S, QADIR J. Deep reinforcement learning for autonomous navigation on duckietown platform: evaluation of adversarial robustness[C]//Proceedings of the 2023 International Symposium on Networks, Computers and Communications. Piscataway: IEEE, 2023: 1-6.
[92] LEE X Y, ESFANDIARI Y, TAN K L, et al. Query-based targeted action-space adversarial policies on deep reinforcement learning agents[C]//Proceedings of the ACM/IEEE 12th International Conference on Cyber-Physical Systems. New York: ACM, 2021: 87-97.
[93] GLEAVE A, DENNIS M, WILD C, et al. Adversarial policies: attacking deep reinforcement learning[J]. arXiv:1905.
10615, 2019.
[94] BOLOOR A, GARIMELLA K, HE X, et al. Attacking vision-based perception in end-to-end autonomous driving models[J]. Journal of Systems Architecture, 2020, 110: 101766.
[95] CAO Y, XIAO C, CYR B, et al. Adversarial sensor attack on lidar-based perception in autonomous driving[C]//Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019: 2267-2281.
[96] WANG C Y, AOUF N. Explainable deep adversarial reinforcement learning approach for robust autonomous driving[J]. IEEE Transactions on Intelligent Vehicles, 2024, PP(99): 1-13.
[97] PINTOR M, DEMETRIO L, SOTGIU A, et al. Detecting attacks against deep reinforcement learning for autonomous driving[C]//Proceedings of the 2023 International Conference on Machine Learning and Cybernetics. Piscataway: IEEE, 2023: 57-62.
[98] ZHU M X, WANG Y H, PU Z Y, et al. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving[J]. Transportation Research Part C: Emerging Technologies, 2020, 117: 102662.
[99] GAO X, LI X Y, LIU Q, et al. Rate GQN: a deviations-reduced decision-making strategy for connected and automated vehicles in mixed autonomy[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 25(1): 613-625.
[100] CHEN Y L, DONG C Y, PALANISAMY P, et al. Attention-based hierarchical deep reinforcement learning for lane change behaviors in autonomous driving[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2019: 1326-1334.
[101] GUPTA D, KLUSCH M. HyLEAR: hybrid deep reinforcement learning and planning for safe and comfortable automated driving[C]//Proceedings of the 2023 IEEE Intelligent Vehicles Symposium. Piscataway: IEEE, 2023: 1-8.
[102] FU Y C, LI C L, YU F R, et al. A decision-making strategy for vehicle autonomous braking in emergency via deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2020, 69(6): 5876-5888.
[103] JOGLEKAR A, SATHE S, MISURATI N, et al. Deep reinforcement learning based adaptation of pure-pursuit path-tracking control for skid-steered vehicles[J]. IFAC-PapersOnLine, 2022, 55(37): 400-407.
[104] CAI P, WANG S, WANG H, et al. Carl-lead: lidar-based end-to-end autonomous driving with contrastive deep reinforcement learning[J]. arXiv:2109.08473, 2021.
[105] MUZAHID A J M, KAMARULZAMAN S F, RAHMAN M A, et al. Deep reinforcement learning-based driving strategy for avoidance of chain collisions and its safety efficiency analysis in autonomous vehicles[J]. IEEE Access, 2022, 10: 43303-43319.
[106] SHI T Y, WANG P, CHENG X X, et al. Driving decision and control for automated lane change behavior based on deep reinforcement learning[C]//Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference. Piscataway: IEEE, 2019: 2895-2900.
[107] CHEN J Y, LI S E, TOMIZUKA M. Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(6): 5068-5078.
[108] WANG H J, GAO H B, YUAN S H, et al. Interpretable decision-making for autonomous vehicles at highway on-ramps with latent space reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2021, 70(9): 8707-8719.
[109] BELLOTTI F, LAZZARONI L, CAPELLO A, et al. Explaining a deep reinforcement learning (DRL)-based automated driving agent in highway simulations[J]. IEEE Access, 2023, 11: 28522-28550.
[110] CUI Z H, LI M, HUANG Y J, et al. An interpretation framework for autonomous vehicles decision-making via SHAP and RF[C]//Proceedings of the 2022 6th CAA International Conference on Vehicular Control and Intelligence. Piscataway: IEEE, 2022: 1-7.