Implementation of Deep Deterministic Policy Gradients for Controlling Dynamic Bipedal Walking

A control system for simulated two-dimensional bipedal walking was developed. The biped model was built based on anthropometric data. At the core of the control is a Deep Deterministic Policy Gradients (DDPG) neural network that is trained in GAZEBO, a physics simulator, to predict the ideal foot location to maintain stable walking under external impulse load. Additional controllers for hip joint movement during stance phase, and ankle joint torque during toe-off, help to stabilize the robot during walking. The simulated robot can walk at a steady pace of approximately 1 m/s, and during locomotion it can maintain stability with a 30 N-s impulse applied at the torso. This work implement DDPG algorithm to solve biped walking control problem. The complexity of DDPG network is decreased through carefully selected state variables and distributed control system.