Abstract
The ability of humans to use visual and tactile information to grasp easily deformable objects and prevent them from deforming and slipping remains a challenge for robotic grasping tasks. The traditional CNN + LSTM network for visual-tactile fusion has the problems of inadequate feature fusion representation and too simple determination of the grasp state category of the object. To solve these problems, this paper proposes a new visual-tactile fusion deep neural network (RSEL) based on the traditional CNN + LSTM network with improvements for evaluating the grasping state of easily deformable objects during grasping. Specifically, we classify the states of easily deformable objects during grasping into five categories: no contact, moderate contact, exce-ssive contact, no slip and slip. In addition, training and testing datasets were built by conducting extensive grasping and lifting experiments on 15 deformable objects with different widths and forces. To evaluate the (RSEL) model, we compared the conventional CNN + LSTM network, and in comparison our model achieved 80.50% classification accuracy with a significant improvement in classification accuracy up to 7.37%. This experiment contributes to adaptive force tuning and robot dexterity operation.