计算机工程与应用 ›› 2018, Vol. 54 ›› Issue (11): 265-270.DOI: 10.3778/j.issn.1002-8331.1708-0308

• 工程与应用 • 上一篇    

交互协调强化学习下的城市交通信号配时决策

夏新海   

  1. 广州航海学院 港航管理学院,广州 510725
  • 出版日期:2018-06-01 发布日期:2018-06-14

Research on urban traffic signal timing decision based on reinforcement learning with interaction coordination mechanism

XIA Xinhai   

  1. School of Port and Shipping Management, Guangzhou Maritime University, Guangzhou 510725, China
  • Online:2018-06-01 Published:2018-06-14

摘要: 针对应用传统强化学习进行城市自适应交通信号配时决策时存在维数灾难和缺乏协调机制等问题,提出引入交互协调机制的强化学习算法。以车均延误为性能指标设计了针对城市交通信号配时决策的独立Q-强化学习算法。在此基础上,通过引入直接交互机制对独立强化学习算法进行了延伸,即相邻交叉口交通信号控制agent间直接交换配时动作和交互点值。通过仿真实验分析表明,引入交互协调机制的强化学习的控制效果明显优于独立强化学习算法,协调更有效,并且其学习算法具有较好的收敛性能,交互点值趋向稳定。

关键词: 交通信号, 交叉口, 协调机制, 强化学习

Abstract: Aiming at the problem of dimensionality curse and the lack of coordination mechanism in the urban adaptive traffic signal timing decision using traditional reinforcement learning, a reinforcement learning algorithm with interaction coordination mechanism is proposed. An independent Q-reinforcement learning algorithm for the urban traffic signal timing decision is designed with the vehicle delay as the performance index. On this basis, the independent Q-reinforcement learning algorithm is extended by introducing interaction coordination mechanism. That is to say, the intersection traffic signal control agent directly exchanges actions and interaction values with the adjacent ones. Simulation results show that the proposed method substantially outperforms the independent reinforcement learning algorithm with more efficient coordination and better convergence performance.

Key words: traffic signal, intersection, coordination mechanism, reinforcement learning