Enhancing Sensory Capabilities: Bringing the Sense of Touch to Robotics. A novel framework to sensorize arbitrary objects using a skin-like soft sensors

Researchers at the Technical University of Munich (TUM) have developed an automatic framework which can fabricate soft sensors via embedded 3D printing that can be wrapped around an object of any shape.

Sensing and perceiving the environment around us are crucial aspects of understanding how we can interact effectively with it. One key factor that significantly influences our interaction with objects is their shape. The shape of an object plays a fundamental role in determining how we can grasp it and manipulate it to accomplish specific tasks. Additionally, the physical properties such as size of the object also have an impact on how we grasp and manipulate it.

When we engage in our daily activities, such as enjoying a beverage, the shape of the cup or glass we use becomes essential. Cups and glasses come in various shapes and sizes, each with its own unique characteristics. As we reach out to grasp and lift a cup, our hands intuitively adjust their grip based on the sensory information they receive. The tactile feedback from our fingers allows us to assess the object's physical properties, i.e., shape, size, and surface texture.

For instance, when encountering different types of cups made from materials like paper, glass, or ceramic, we adapt our grasp based on our perception of the object's properties. The shape of the cup influences how we position our fingers and wrap our hand around it. The size of the cup guides us in choosing the appropriate finger placement and grip strength required to hold it comfortably.

Moreover, the compliance of the cup or glass, which refers to its flexibility or rigidity, affects our grasp and manipulation. A rigid glass may require a firm grip to ensure stability, while a more flexible cup, such as a paper or plastic one, may allow for a more relaxed grip. The compliance of the object provides sensory cues that inform our grasp, helping us gauge how much force is required and how the object will respond to our manipulation.

These adaptations in our grasp based on the shape, size, and compliance of the object reflect our ability to perceive and adjust our actions to achieve successful interactions with the environment. Our sensory feedback guides us in real-time, enabling us to effortlessly and precisely manipulate objects in our daily tasks. This intricate interplay between our senses, motor skills, and environmental properties highlights the remarkable capabilities of the human sensorimotor system.

In the field of robotics and prosthetics, the integration of sensors, particularly force and torque sensors, has been a common approach to emulate sensory systems. These sensors provide valuable feedback on the interaction forces and torques between the robotic or prosthetic device and the environment. However, the integration of traditional sensors presents certain limitations, particularly when it comes to customization and integration with different surfaces and objects.

To overcome these challenges, researchers have been exploring the use of soft sensors in recent years. Soft sensors offer advantages in terms of flexibility, adaptability, and ease of integration into various surfaces and objects. Unlike traditional sensors, which are often rigid and have limited conformability, soft sensors can be designed and fabricated to conform to the shape and surface of objects, enabling more versatile and customised sensing capabilities.

While some research laboratories have been actively working on developing soft sensors, there has been a lack of an end-to-end framework that allows for sensorizing any object, irrespective of its shape or size. This means that the current efforts have been more focused on developing specific soft sensors for particular applications or objects, rather than providing a universal solution.

The development of an end-to-end framework for sensorizing any object would be a significant breakthrough. Such a framework would encompass the design, fabrication, and integration of soft sensors onto a wide range of objects, regardless of their shape, size, or material composition. It would enable the creation of a highly adaptable and customizable sensing system, opening up new possibilities in robotics, prosthetics, and human-machine interaction.

This kind of framework would require advancements in materials science, sensor fabrication techniques, and integration methodologies. It would involve the development of soft and stretchable materials that can house sensing elements, as well as the integration of signal processing and data interpretation algorithms to make sense of the sensor data in real-time.

The realisation of an end-to-end framework for universal sensorization of objects would have transformative implications. It would enable robots and prosthetic devices to have a more comprehensive understanding of their surroundings, allowing for more dexterous and context-aware interactions. It would also pave the way for applications in areas such as object recognition, grasping and manipulation, and human-robot collaboration, enhancing the overall performance and usability of these systems.

In their innovative research, Sonja Groß, Diego Hidalgo, Abdeldjallil Naceri, and Amartya Ganguly collaborated to develop an advanced framework. This framework addresses the challenge of integrating sensors onto various 3D shapes using soft skin-like materials.

Traditionally, integrating sensors onto complex 3D objects has been a cumbersome and intricate task. However, the research team's groundbreaking framework provides an automatic solution that simplifies the process. By leveraging their expertise in robotics and machine intelligence, they successfully fabricated soft skin-like sensors that can be easily wrapped around any 3D object.

The framework developed by the team allows for the customization of sensors according to the specific shape and contours of the object. This flexibility enables the sensors to conform to the object's surface, capturing precise and accurate data about its interactions with the environment.

By using soft skin-like materials, the sensors exhibit excellent flexibility and conformability. They are capable of adapting to objects with various shapes, sizes, and materials. This versatility expands the possibilities for sensorization in robotics, prosthetics, and other related fields.

The integration of these soft skin-like sensors onto 3D objects opens up new avenues for advanced sensing capabilities. The sensors can provide valuable information about contact forces, deformations, and other tactile feedback, enhancing the object's perception and enabling more sophisticated interactions.

The research team's automatic framework represents a significant step forward in the field of sensor integration onto complex 3D objects. Their work has the potential to revolutionise industries such as robotics, prosthetics, and human-machine interaction, by enabling seamless and customizable sensorization on a wide range of objects.

This work was recently presented at ICRA 2023, London, United Kingdom.


S. Groß, D. Hidalgo, S. Breimann, N. Stein, A. Ganguly, D. Naceri, S. Haddadin; Soft Sensing Skin for Arbitrary Objects: An Automatic Framework, IEEE International Conference on Robotics and Automation (ICRA), 2023

Receive our featured stories, news, events, and videos in your inbox every month, subscribe here!