3D Visual-guided Robot Arm Control for A Warehouse Automation System

Warehouse automation is greatly beneficial in improving a wide variety of industries. However, the prevalent automation methods apply in industrial fields where systems are difficult to initialize and it is hard to recognize the system status. In this work, a 3D visual-guided robot arm system with marker detection and object detection is proposed. There are two main parts in this study, including system initialization and validation using marker detection and storage and retrieval using magazine detection. The system is composed of two cameras for the stereo system, a robot arm, and computer vision algorithms to form the system for detecting, classifying, and picking objects by a robot arm. Besides, magazines that can store items such as nuts and bolts and a frame that can store magazines into its grids are used. Firstly, the system is initialized by marker detection method which detects marker positions on a frame and saves frame and grid positions where the robot arm can approach to store or retrieve magazines. After that, using contour detection of deep learning method and Hough line transform, the correct magazine center position in a grid can be estimated. If an impact occurs such as an earthquake, the warehouse system must check the status to see if the system can be run perfectly. This study introduces solutions which avoid the above problem. The work also shows an error under 1mm between magazine position and grid position.