What did you find out?
In this new publication, we focus on how haptic sensors and human stiffness estimation can improve teleoperation performance. More and more robots are being designed to work interactively with people and in complex environments. Collaboration with humans requires robots to respond quickly and intelligently, much like another human. The “variable impedance control” central to this work helps the robot adapt according to the task or environment. This is typically achieved through “Learning from Demonstration,” where the robot records, learns, and generalizes movement parameters from human operators to adapt to similar tasks. In this research, we estimate the “end-point stiffness” of the human arm using a haptic device instead of relying on electromyographic (EMG) measurements. Our approach involves creating a dataset for the Learning from Demonstration process. The motivation behind our work is to replace EMG devices with haptic input devices, which are essential for stiffness estimation in teleoperation. We used the collected data to design a variable impedance controller for specific tasks. Our results show that the estimated stiffness data can effectively reflect changes in the environment during pushing and releasing tasks. Notably, our controller designed with this dataset performs better and exhibits less force signal distortion, even with a time-delayed network, compared to an EMG-based solution.
What challenges did you face during the research?
- Unlike many other studies that estimate human stiffness using EMG measurements, we had to estimate stiffness indirectly using our haptic devices and sensors on the robot arm. This required developing a novel dynamic solution with force and acceleration data.
- The new variable impedance controller needed to be trained with a small dataset, as large-scale parallel experiments are challenging and time-consuming to conduct.
- Another ongoing challenge is assessing the subjective performance of the controller, which relates to the quality of the user’s experience.
What are the practical implications of your research?
The research on variable impedance control and learning from demonstration in teleoperation can be effectively applied in industrial scenarios where humans collaborate with robots or where the environment changes during tasks. For example, a human operator can teach a robot to polish metal surfaces or assemble parts. These tasks require the robot to mimic human behavior and adjust impedance accordingly. With our new controller and a control policy learned through “Learning from Demonstration,” the robot can handle tasks even when settings, such as part shapes or positions, change. Our findings also enhance the quality of feedback forces during teleoperation. Without variable impedance control, the remote robot would experience sudden force changes when contacting other objects. The variable impedance controller, however, ensures smoother and more stable feedback forces.
What unique challenges did you face during your research?
- Unlike many other research studies where human stiffness is estimated with EMG devices, we have to indirectly estimate human stiffness using the haptic device and the different sensors on the robot arm. This challenge requires developing a novel dynamic solution with force and acceleration data.
- The control policy of the variable impedance controller must be learned with a small dataset, as large-scale parallel experiments are challenging and time-consuming to conduct.
- Assessing the subjective performance of the controller, which stands for the quality of experience of the human user, is another challenge we are still working on.
What are your new findings suitable for?
The research on variable impedance control and learning from a demonstration of teleoperation can be widely utilized in industrial scenarios where people collaborate with robots or when the environment changes during tasks. For example, the human operator can teach the robot to polish the metal surface or assemble parts. These tasks require the robot to mimic the human’s behavior and change the control impedance. When the robot is equipped with our new controller and a control policy learned with “Learning from Demonstration”, the robot still executes the tasks when some settings, such as the shapes or the positions of the parts, are changed. The findings can also benefit the feedback force quality of the teleoperation. Without the variable impedance controller, the remote robot will generate sudden force changes in contact with other objects. On the other hand, the variable impedance controller can change the control impedance to make the feedback force smoother and more stable.
Paper
Zican Wang; Xiao Xu; Dong Yang; Basak Güleçyüz; Fanle Meng; Eckehard Steinbach
Teleoperation with Haptic Sensor-aided Variable Impedance Control Based on Environment and Human Stiffness Estimation
IEEE Sensors Journal, 3-2024