Abstract: This paper proposes a method of building energy conservation based on parallel reinforcement learning, which combines a multi-threading technique and the experiment replay method to propose a multi-threading parallel reinforcement learning algorithm framework. Its novelty lies in that it introduces the self-simulation metric method in the experience replay process, which selects samples with low similarity by calculating the distance between samples to construct multiple sample pool. And in the learning process, Agent selects samples from the multiple sample pool to effectively avoid wasting learning resources. The experiments include the comparison experiments with the Q-Learning algorithm and the PID method on the simulation room model. Experimental results show that the proposed parallel algorithm has faster learning rate and convergence rate, and it can solve the optimal policy faster with higher operating efficiency.

Key words: reinforcement learning, parallel reinforcement learning, experiment replay, multi-threading technology, building conversation

摘要： 提出一种基于并行强化学习的建筑节能方法，该方法结合多线程技术和经验回放方法提出一个多线程并行强化学习算法框架，其新颖点在于：在经验回放过程中引入自模拟度量方法，通过计算样本之间的距离，选取低相似度的样本构造多样样本池，Agent的学习过程从多样样本池中选取样本学习，可有效避免浪费学习资源。实验包括在仿真房间模型上与Q-Learning算法的对比实验和与经典PID控制方法的对比实验。实验结果表明，所提出的并行算法有更快的学习速率和收敛速度，能更快地求解出最优策略，并拥有更高的运行效率。

关键词: 强化学习, 并行强化学习, 经验回放, 多线程技术, 建筑节能