关键词: 平均奖赏, 强化学习, R-学习算法, 学习分类元系统（LCS）, 零阶分类元系统（ZCS）, 多步学习问题

Abstract: As a genetics-based machine learning technique, Zeroth-level Classifier System（ZCS） has shown promise in applying to multi-step problems. However, the standard ZCS is based on a discounted reward reinforcement learning algorithm, which optimizes the discounted total reward received by an agent but is not suitable for all multi-step problems. There are some average reward reinforcement learning methods available, such as R-learning, which optimize the average reward per time step. In this paper, R-learning is used as the reinforcement learning employed by ZCS, to replace its discounted reward reinforcement learning approach. The modification results show classifier system can effectively prevent the occurrence of overgeneralization and support long action chains, and thus is able to solve large multi-step problems.

Key words: average reward, reinforcement learning, R-learning, Learning Classifier Systems（LCS）, Zeroth-level Classifier System（ZCS）, multi-step problems