Abstract: Aiming at the problem of dimensionality curse and the lack of coordination mechanism in the urban adaptive traffic signal timing decision using traditional reinforcement learning, a reinforcement learning algorithm with interaction coordination mechanism is proposed. An independent Q-reinforcement learning algorithm for the urban traffic signal timing decision is designed with the vehicle delay as the performance index. On this basis, the independent Q-reinforcement learning algorithm is extended by introducing interaction coordination mechanism. That is to say, the intersection traffic signal control agent directly exchanges actions and interaction values with the adjacent ones. Simulation results show that the proposed method substantially outperforms the independent reinforcement learning algorithm with more efficient coordination and better convergence performance.

Key words: traffic signal, intersection, coordination mechanism, reinforcement learning

摘要： 针对应用传统强化学习进行城市自适应交通信号配时决策时存在维数灾难和缺乏协调机制等问题，提出引入交互协调机制的强化学习算法。以车均延误为性能指标设计了针对城市交通信号配时决策的独立Q-强化学习算法。在此基础上，通过引入直接交互机制对独立强化学习算法进行了延伸，即相邻交叉口交通信号控制agent间直接交换配时动作和交互点值。通过仿真实验分析表明，引入交互协调机制的强化学习的控制效果明显优于独立强化学习算法，协调更有效，并且其学习算法具有较好的收敛性能，交互点值趋向稳定。

关键词: 交通信号, 交叉口, 协调机制, 强化学习