Simulation of Assisted Human Walking Using Musculoskeletal Model Coupled with Exoskeleton via Deep Reinforcement Learning

This paper presents a novel approach to simulate assisted human walking to evaluate the assistance of the powered lower-limb exoskeleton. We first construct a subject specific musculoskeletal model as an agent that generates muscle forces according to internal and external states, and describe the exoskeleton as a rigid multi-link structure with compensatory torques applied at the joints. Then we train the agent to produce walking motion using a deep reinforcement learning algorithm given recorded experimental data on a biomechanical simulator. Next, we combine the pre-trained musculoskeletal agent with the exoskeleton and perform the assisted walking simulation which takes into account the human exoskeleton dynamics. Simulated energy expenditures under different experimental conditions are compared to evaluate the assistance effectiveness of the exoskeleton. Results show that the proposed method has great potential in providing insights into the human-exoskeleton interaction.