Deep-Learning 을 활용한 로봇의 Interaction Force 예측(삼성전자 미래육성센터2017-2020)
(Inferring Interaction Force of Robot from only Visual Information without Force/Torque Sensor)
- 영상 기반의 입력 가능한 digital pen(교육부, 2018-2019)
- 촉각 센서를 활용한 원격 수술로봇/원격 손 제어 (산업통상자원부, 2017-2020)
- Proprioception(고유수용성감각) 측정을 위한 장치 설계 및 VR 활용 실험 (보건복지부,국립재활원, 2017)
- Visual-tactile sensory perception (교육부, 2016-2018)
- 360도 영상 촬영을 위한 원격 조종로봇 개발(문화체육관광부, 2016)
- VR 착용감 개선을 위한 얼굴 감각 인지능력 측정(삼성전자, 2016)
- Deep-Learning 을 활용한 로봇의 Interaction Force 예측(삼성전자 미래육성센터2017-2020)
(Inferring Interaction Force of Robot from only Visual Information without Force/Torque Sensor)
– 영상 및 로봇 움직임에 기반한 학습을 통해 로봇과 환경과의 Interaction force를 예측하고 이를 기반으로 한 로봇 원격조종 햅틱 피드백
: 로봇이 물체와 상호작용 시 생성되는 상호작용 힘 관련 DB 생성
: 로봇 원격 제어 또는 자율 움직임 시 환경과의 상호작용 힘을 영상(RGB 또는 RGB-D) 및 로봇의 움직임정보와 모터의 전류 정보를 기반으로 물리적 감각 재현 예측 모듈 연구
: 예측된 힘을 원격제어에 활용한 햅틱 피드백 시스템
“Vision-Based Interaction Force Estimation of Robot without Tactile and Force_Torque Sensor”, IEEE Industrial Informatics, Under review
“Sequential Image-based Attention Network for Inferring Force Estimation without Haptic Sensor”, IEEE Access, early access, 2019, DOI:10.1109/ACCESS.2019.2947090
“An Efficient Three-Dimensional Convolutional Neural Network for Inferring Physical Interaction Force from Video”, Sensors, 19(16), 3579, 2019, doi : 10.3390/s19163579
“Interaction Force Estimation using Camera and Electrical Current without Force/Torque Sensor”. IEEE Sensors Journal, Vol 18, No. 21, pp 8863-8872, 2018, doi: 10.1109/JSEN.2018.2868332
“Inferring Interaction Force from Visual Information without Using Physical Force Sensors”, Sensors, 17(11), 2455, 2017, doi: 10.3390/s17112455
- 로봇 원격제어시 물리적 입력값을 활용한 시간지연없는 영상 생성 기법 개발(과학기술부-중견연구, 2020~2023)
– 사용자가 로봇 원격조종 시 네트워크 및 물리적 거리의 한계를 극복하고 영상을 전달하기 위해, 원격지에서 사용되는 큰 이미지의 전송 빈도수를 줄여 전송한 지연된 영상정보, 지연되서 받는 로봇의 상호작용 힘 정보 및 원격 조작 정보를 조종자의 조종입력값과 같이 사용하여딥러닝 기법을 활용한 실시간 영상 생성 및 상호작용 힘 생성 방법을 개발하고 이 영상 및 힘 정보를 사용자에게 피드백 하여 원격로봇 제어함
“Continuous Image Generation from Low-Update-Rate Images and Physical Sensors through a Conditional GAN for Robot Teleoperation”, IEEE Transactions on Industrial Informatics, 2020, Accepted, DOI: 10.1109/TII.2020.2991764
- 고유수용성 감각의 측정 및 사용자 힘증강을 위한 Exoskeleton 개발
- EMG/EEG 신호를 활용한 Interaction force 예측
- 딥러닝을 사용한 공공기술의 기술사업화 여부 판단 모델 개발(KISTI, 2020)
PAST
- 영상 기반의 입력 가능한 digital pen(교육부, 2018-2019)
- 촉각 센서를 활용한 원격 수술로봇/원격 손 제어 (산업통상자원부, 2017-2020)
- Proprioception(고유수용성감각) 측정을 위한 장치 설계 및 VR 활용 실험 (보건복지부,국립재활원, 2017)
- Visual-tactile sensory perception (교육부, 2016-2018)
- 360도 영상 촬영을 위한 원격 조종로봇 개발(문화체육관광부, 2016)
- VR 착용감 개선을 위한 얼굴 감각 인지능력 측정(삼성전자, 2016)
- Haptics in Surgical Robot
When performing open surgery, surgeons use their touch sense intuitively in combination to get the information. However, during the minimally invasive laparoscopic robotic surgery with the conventional surgical robot, this haptic information can be lost because of the absence of force sensors in robot tools and haptic feedback instrument in master system. Sensors in the slave system and a combine tactile and kinesthetic feedback apparatus in the master system was developed to provide touch sense to surgeon in haptic feedback system of surgical robot. The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon’s ability to control the interaction force applied to the tissue.
– “Grip force measurement of forceps with fiber Bragg grating sensors”, Electronics Letters, vol 50, no 10, pp.733-735, 2014 (SCI) ※ This paper was selected in the feature section in the front of Electronics Letters (vol 50, no 10)
– “Tactile Display with Tangential and Normal Skin Displacement for Robot-Assisted Surgery,” Advanced Robotics, vol 28, Issue 13, pp 859-868, 2014
– “Development of Flexible Three-Axis Tactile Sensor Based on Screen-Printed Carbon Nanotube-Polymer Composite”, Journal of Micromechanics and Microengineering, vol 24, no 7
– “Role of combined tactile and kinesthetic feedback in minimally invasive surgery”, International Journal of Medical Robotics and Computer Assisted Surgery, Vol 11, Issue 3, pp 360–374, 2015
– “Three-axis pneumatic tactile display with integrated capacitive sensors for feedback control”, Microsystem Technologies, Volume 22, Issue 2, pp 275-282
– “Position controlled pneumatic tactile display for tangential stimulation of a finger pad”, Sensors and Actuators A: Physical, Volume 229, pp 15–22, 2015
- SmartWatch Interface
Touchscreen interaction has become a fundamental means of controlling mobile phones and smartwatches. However, the small form factor of a smartwatch limits the available interactive surface area. To overcome this limitation, we propose the expansion of the touch region of the screen to the back of the user’s hand. We developed a touch module for sensing the touched finger position on the back of the hand using infrared (ir) line image sensors, based on the calibrated ir intensity and the maximum intensity region of an ir array.
– “Expansion of Smartwatch Touch Interface from Touchscreen to Around Device Interface Using Infrared Line Image Sensors”, Sensors, 15(7), pp16642-16653, 2015
Inspired by the mechanisms of bone conduction transmission, we present a novel sensor and actuation system that enables a smartwatch to securely communicate with a peripheral touch device, such as a smartphone. Our system regards hand structures as a mechanical waveguide that transmits particular signals through mechanical waves. As a signal, we used high-frequency vibrations (18.0–20.0 khz) so that users cannot sense the signals either tactually or audibly. To this end, we adopted a commercial surface transducer, which is originally developed as a bone-conduction actuator, for mechanical signal generation.
– “Transferring Data from Smartwatch to Smartphone through Mechanical Wave Propagation”, Sensors, 15(9), pp 21394-21406, 2015
- Haptics (Interface)
- Hip Force Exoskeleton
- 360도 3D 카메라 촬영을 위한 모바일 로봇
- VR 활용 3D CAD를 위한 3차원 Interface
- VR 장치를 위한 얼굴 민감도 측정
- Haptics (psycho-physical experiment)
Beat perception on the Finger with two pin
Two vibrations with slightly diVerent frequencies induce the beats phenomenon. In tactile perception, when two pins of diVerent frequencies stimulate the Wngertips, an individual perceives a beats caused by a summation stimulus of the two vibrations. The present study demonstrates experimentally that humans can perceive another vibration based on the beats phenomenon when two tactile stimuli with slightly diVerent frequencies are stimulated on the Wnger pad with a small contactor in diVerent locations at the same time. Moreover, we examined the amplitude of the detection threshold to be able to perceive beats phenomenon on the index Wnger with 5 carrier frequency (63.1, 100, 158.5, 251.2, and 398.1 Hz) and 4 beats frequency (2.5, 3.98, 6.31, and 10 Hz) when two stimuli 1 mm distance apart are vibrated at a slightly diVerent frequency. From the experiments, it is concluded that the amplitude threshold to be able to perceive beats decreases as the standard frequency increases under 398 Hz. Furthermore, from comparing the absolute detection threshold and beats detection threshold, as the carrier frequency increases, the required amplitude at two pins for the detection of beats decreases compared to absolute vibration.
– “Effect of Frequency Difference on Sensitivity of Beat Perception.” Experimental Brain Research, vol 216, pp. 11-19, 2012
Tactile Apparent Motion
The effects of the frequency modulation of vibration elements on the representation of dynamic tactile apparent motion between both hands will be proposed. The sensation level difference due to the different frequencies that result when using vibrating motors on the right and left fingers causes a phantom sensation that is perceived as if the stimuli were between the fingers. The change of sensation level difference between both hands due to the frequency modulation creates a somatosensory illusion using this phantom sensation, which occurs in such a way as to feel like a vibration flow from one hand to the other hand.
– “Tactile apparent motion between both hands based on frequency modulation,” in haptics: perception, devices, mobility, and communication, springer, 2012, pp.293-300
Shape Perception with frequency modulation
This study attempted to observe what effects the frequency modulation of vibration elements produce in representing a tactile shape. Tactile shapes were modulated based on frequency difference at constant amplitude through a tactile feedback array of 30 (5 × 6) pins, which stimulated the finger pad.
– “Presentation of Surface Height Profiles Based on Frequency Modulation at Constant Amplitude Using Vibrotactile Elements.” Advanced Robotics, vol 25, Issue 16, pp. 2065-2081, 2011