Journal Paper Accepted (IEEE Sensors Journal): Interaction Force Estimation using Camera and Electrical Current without Force/Torque Sensor

In this paper, we propose a method for the estimation of the interaction forces between the motorized system and object through visual and electric information. In particular, we propose a new interaction force sensing method based on sequential images and the electrical current from the motor during the interaction between the system and environment to estimate the interaction force using deep learning. In the previous method, to measure the interaction force using only visual information, the prediction is inaccurate when the system interacts with an undeformable target, even though the aspect of the change appears small in the image. We use a neural network structure for estimating the interaction force from the time-series data of visual and electric information using deep learning, which combines the convolution neural network and long short-term memory models. From the evaluation to show the feasibility of the interaction force estimation, the proposed learning models successfully estimate the forces for four targets (rigid box, rigid box on sponge, sponge, and stapler), which are both deformable and undeformable objects. The proposed method demonstrates the best results in the interaction force estimation between the motorized system and object.

Published in: IEEE Sensors Journal ( Early Access )