Obstacle rearrangement for robotic manipulation in clutter using a deep Q-network

We propose a learning-based method to solve the problem of rearranging objects in clutter to obtain a collision-free path for retrieving a target object by a robotic manipulator. The method provides the solutions for what obstacles to remove and where to place them in what orders to rearrange the obstacles. The proposed method uses a deep Q-network to learn the optimal policies for robot’s rearranging actions. To apply the network, it is assumed that the configurations of objects and environment are known and the environment is considered as a grid space. Two types of structures for a deep Q-network are proposed according to action characteristics for this problem. From extensive simulation experiments, we showed that our algorithm could reduce the number of rearranged obstacles and the total execution time significantly (up to 35%) compared to a baseline method. The experiments were performed by a real robot with a vision system and showed the feasibility of the proposed method on real world.