Visual-Guided Robot Arm Using Multi-Task Faster R-CNN

The limitation of current visual recognition methods is a big obstacle for the application of automated robot arm systems into industrial projects, which require high precision and speed. In this work, we present a Faster RCNN based multi-task network, a deep neural network model, that is able to simultaneously perform three tasks including object detection, category classification and object angle estimation. Afterward, the outputs of all three tasks are utilized to decide a picking point and a rotated gripper angle for the pick-and-place robot arm system. The test results show that our network achieves a mean average precision of 86.6% at IoU (Intersection over Union) of 0.7, and a mean accuracy of 83.5% for the final prediction including object localization and angle estimation. In addition, the proposed multi-task network takes approximately 0.072 seconds to process an image, which is acceptable for pick-and-place robot arms.