MultiTip

Royal Society

Advancing Robot Dexterity with Multimodal Vision-Based Tactile Sensing

2024 - 2027

Summary

In today’s fast-paced industries, where repetitive tasks are common, there is a growing need to relieve humans from monotonous jobs while maintaining high productivity. The MultiTip project aims to revolutionize robotics by enhancing how robots sense and manipulate objects with advanced tactile technology.

At the heart of this initiative is the development of MultiTip, a new tactile sensor that mimics human touch. This sensor enables robots to detect both slow and fast skin deformations, allowing them to perceive subtle changes in contact with objects, like human fingertips can perform. Such capabilities are crucial for tasks requiring delicacy and precision, such as assembling electronic components.

The project will integrate various technologies, including accelerometers for vibration detection and Inertial Measurement Units for pose tracking, into the MultiTip sensor. By using a control model trained on a broad spectrum of simulated tactile signals, the transferred model will enable real robot manipulators to interact with objects more agilely. This advancement will allow for manipulations previously only achievable by humans.

Objectives

WP1 – MultiTip with High-frequency and Proprioceptive Sensory Integration. We will elevate the VBTS capabilities to human finger-level by integrating multiple tactile modalities, including accelerometers for vibrations and an inertial measurement system to track the sensor’s poses (translations and rotations) over time.

WP2 – Skin Redesign and Dynamic Modelling. To create a virtual simulator, it will be crucial to accurately model the sensor’s deformation and vibration in response to external forces and ensure consistent fabrication quality for the sensor, achieved with a novel 3D-printed flexible mesh-link structure to support the tactile skin.

WP3 – Tactile Data-based Object Manipulations via Sim2Real Transfer. We aim to show object manipulation skills that has been challenging with VBTS based only on SA-type signal sensing, by using the outcomes developed in WP1 and WP2 along with the latest AI techniques that have been trialled in the hosting institution.

Related publications include:

Project Partners and Funding

The place of tenure of this award is the School of Engineering Mathematics and Technology (SEMT), University of Bristol and the Graduate School of Data Science, Kyungpook National University. The award has been made to support the collaboration of Dr Efi Psomopoulou at University of Bristol and Dr Saekwang Nam at Kyungpook National University. The award is funded by the Royal Society and the National Research Foundation of Korea.