Abstract: In modern processors, memory controller is the manager and executant of the access to off-chip memories. The scheduling algorithm used by memory controller has a great influence on the performance of access to off-chip memory. For the situation that current scheduling algorithm can’t adapt to the change of workload, it proposes ALHS algorithm based on reinforcement learning method in this paper. ALHS optimizes its scheduling policy by adjusting the upper limit of page hits in FR-FCFS scheduling. The experiment on multi-workload shows that ALHS improves access speed by 10.98% compared to FR-FCFS and obtains performance improvement close to the optimistic policy. It shows that ALHS can explore the experiment environment and optimizes itself.

Key words: memory controller, memory scheduling algorithm, adaptive, reinforcement learning

摘要： 在现代处理器中，存储控制器是处理器芯片对片外存储器进行访问的管理者和执行者，其中对访存过程的调度算法会对实际访存性能产生十分重要的影响。针对已有调度算法在不同负载特征下自适应性不足的问题，提出了一种基于强化学习方法的ALHS算法，通过对访存调度中页命中优先时的连续页命中上限次数进行自适应调整，习得最优策略。多种不同典型访存模式的模拟结果显示，相比传统的FR-FCFS，ALHS算法运行速度平均提升了10.98%，并且可以获得近似于最优策略的性能提升，表明该算法能够自主探索环境并自我优化。

关键词: 存储控制器, 访存调度算法, 自适应性, 强化学习