关键词: 分层强化学习, Option, 蚁群聚类算法, 经验回放

Abstract: A new algorithm for Option automatic generation of hierarchical reinforcement learning is presented.The algorithm takes the state space explored by Agent as input in the initial learning phase and clusters the states employing Ant Colony Clustering Algorithm（ACCA）.Based on the clustered state sets，the intra-strategies are learned by an experience replay procedure.As a result，the Options are generated.The validity of the algorithm is demonstrated by simulation experiments.

Key words: hierarchical reinforcement learning, Option, Ant Colony Clustering Algorithm（ACCA）, experience replay