Direct-Vision-Based Reinforcement Learning in “Going to a Target” Task with an Obstacle and with a Variety of Target Sizes

Two of us has proposed a direct-vision-based reinforcement learning on the neural-network system, in which raw visual sensory signals are directly used as the inputs of the neural network. It has been shown that it has the ability to integrate many local sensory signals, to obtain a smooth evaluation function and then to generate appropriate motions. In this previous work, simple “going to a target” task was chosen as an example. Here we apply it to more difficult tasks to evaluate the effectiveness of the direct-vision-based reinforcement learning. In the first task, the object size is varied at every trial. After the reinforcement learning, the robot was able to obtain an appropriate evaluation function that scarcely depended on the target size. In the second task, an obstacle is located in the “going to a target” task, and the obstacle location is varied at every trial. By employing two kinds of visual sensors, such that one of them can catch only the target object and the other can catch only the obstacle, the robot became to obtain the motions to avoid the obstacle and to go towards the target object. The hidden neurons in the reinforcement learning was applied to another supervised learning. It was then shown that the spatial information obtained in hidden neurons was useful in, in other words, can be succeeded to another learning.