关键词: 蛇形机器人, 万向结构, 强化学习, 近端策略优化算法（PPO）

Abstract: In order to solve the problems of complex structure and insufficient flexibility of the snake-like robot, a snake-like robot structure with cross shaft universal joint is proposed. The snake-like robot is composed of 6 modules, each module is equipped with a passive wheel, and the motor drives the slider on the ball screw to move, so that the connecting rod drives the universal joint to deflect, so as to realize the meandering movement and ensure the flexibility of the robot’s movement. At the same time, a control scheme based on deep reinforcement learning is presented for the complex problem of snake robot modeling. The MuJoCo physics engine is used to build an interactive environment for learning, and the proximal policy optimization（PPO） algorithm is adopted to train the optimal motion strategy to guide the action. After using the proposed model for learning and training, the simulation experimental data show that the motion strategy trained by the PPO algorithm can complete the straight forward motion in the environment of different friction coefficients, which showing that it has a certain adaptability for different terrain environments. Finally, the feasibility and stability of this scheme are verified by prototype physical test experiments.

Key words: snake-like robot, universal structure, reinforcement learning, proximal policy optimization（PPO）