Computer Engineering and Applications ›› 2021, Vol. 57 ›› Issue (3): 72-79.DOI: 10.3778/j.issn.1002-8331.2010-0038

Previous Articles     Next Articles

Overview of Hierarchical Reinforcement Learning

LAI Jun, WEI Jingyi, CHEN Xiliang   

  1. College of Command Information System, Army Engineering University, Nanjing 210007, China
  • Online:2021-02-01 Published:2021-01-29



  1. 陆军工程大学 指挥控制工程学院,南京 210007


In recent years, reinforcement learning has increasingly reflected its strong learning ability. In 2017, AlphaGo beat the world champion in go. Meanwhile, in the complex competitive games StarCraft 2 and dota2, the top human teams are also defeated by AI. However, it has its own weaknesses, and the bottleneck gradually appears in the continuous development. Hierarchical reinforcement learning can solve the problem of dimension disaster, which makes it show more excellent processing ability in the environment with more complex environment and larger action space. This paper briefly introduces the basic theory of reinforcement learning. It introduces three classical hierarchical reinforcement learning algorithms, option, hams and MAXQ. It summarizes and analyzes the hierarchical reinforcement learning algorithm proposed in recent years under the idea of stratification from three aspects. It discusses the development prospects and challenges of hierarchical reinforcement learning.

Key words: hierarchical reinforcement learning, subpolicy sharing, multi-layer hierarchical structure, automatic stratification



关键词: 分层强化学习, 子策略共享, 多层分层结构, 自动分层