ACMViz: a visual analytics approach to understand DRL-based autonomous control model

Deep reinforcement learning (DRL) has been widely used in autonomous control due to its superior performance. DRL-based autonomous control model (ACM) aims to train an agent to achieve self-control and learn optimal policy through pre-defined rewards. Despite the super-human performance, ACM is regarded as a black box, and the interpretation of its internal working mechanism remains a challenge to domain experts. In addition, adjusting the reward settings of ACM is also challenging due to the uncertain relationship between rewards setting and strategies. In this paper, we propose ACMViz, a visual analytics system to explore control strategies at different stages and reveal the relationship between rewards and action patterns. Focusing on controlling a lunar lander, ACMViz investigates different landing trajectories and action sequences to interpret the model and control the training. From our visual analytics of the action patterns, we diagnose and improve reward settings for different control targets. Through our case studies with deep learning experts, we validate the effectiveness of ACMViz.