A Validation Approach for Deep Reinforcement Learning of a Robotic Arm in a 3D Simulated Environment

In recent years, deep reinforcement learning has increasingly contributed to the development of robotic applications and boosted research in robotics. Deep learning and model-free, off-policy, value-based reinforcement learning algorithms enabled agents to successfully learn complex robotic skills through trial and error process and visual inputs. The aim of this paper concerns the training of a robot in a simulation environment by designing a Deep Q-Network (DQN) that elaborates images acquired by an RGB vision sensor inside a 3D simulated environment and outputs a value for each action the robotic arm can execute given the current state. In particular, the robot has to push a ball into a soccer net without any knowledge of the environment and its own location. In addition, our further goal was to perform agent validation during training and assess its generalization level. Despite the many advances in reinforcement learning, it is still a challenge. Therefore, we devised a validation strategy similar to the method applied in supervised learning and tested the agent both on known and unknown experiences, achieving interesting and promising results.