Online Shape Estimation based on Tactile Sensing and Deformation Modeling for Robot Manipulation

Precise robot manipulation of deformable objects requires an accurate and fast estimation of their shape as they deform. So far, visual sensing has been mostly used to solve this issue, but vision sensors are sensitive to occlusions, which might be inevitable when manipulating an object with robot. To address this issue, we present a modular pipeline to track the shape of a soft object in an online manner by coupling tactile sensing with a deformation model. Using a model of a tactile sensor, we compute the magnitude and location of a contact force and apply it as an external force to the deformation model. The deformation model then updates the nodal positions of a mesh that describes the shape of the deformable object. The proposed sensor model and pipeline, are evaluated using a Shadow Dexterous Hand equipped with BioTac sensors on its fingertips and an RGB-D sensor.