Manipulation of Polymorphic Objects Using Two Robotic Arms through CNN Networks

This article presents an interaction system for two 5 DOF (Degrees of Freedom) manipulators with 3-finger grippers, which will be used to grab and displace up to 10 polymorphic objects shaped as pentominoes, inside a VRML (Virtual Reality Modeling Language) environment, by performing element detection and classification using an R-CNN (Region Proposal Convolutional Neural Network), and point detection and gripping orientation using a DAG-CNN (Directed Acyclic Graph-Convolutional Neural Network). It was analyzed the feasibility or not of a grasp is determined depending on how the geometry of an element fits the free space between the gripper fingers. A database was created to be used as training data with each of the grasp positions for the polyshapes, so the network training can be focused on finding the desired grasp positions, enabling any other grasp found to be considered a feasible grasp, and eliminating the need to find additional better grasp points, changing the shape, inclination and angle of rotation. Under varying test conditions, the test successfully achieved gripping of each object with one manipulator and passing it to the second manipulator as part of the grouping process, in the opposite end of the work area, using an R-CNN and a DAG-CNN, with an accuracy of 95.5% and 98.8%, respectively, and performing a geometric analysis of the objects to determine the displacement and rotation required by the gripper for each individual grip.