LingBot-Touch Aims to Unify Data From Robot Touch Sensors

LingBot-Touch is designed to make touch data from different kinds of robot sensors usable in a common format, giving manipulation systems a shared representation to work with. The model was presented in a research paper posted Nov. 9 and was tested in simulation and on real robots.

Robotic touch sensors can produce different forms of data. LingBot-Touch handles inputs from piezoresistive, magnetic and visuotactile sensors, encoding them in a shared latent space. The aim is to make tactile representations usable across hardware, rather than tying them to one sensor type.

The model uses a shared Transformer to predict latent representations from spatiotemporal tactile context. Those representations become tokens that can be plugged into different manipulation systems, including diffusion-policy models and vision-language-action models. The paper identifies π₀.₅ and LingBot-VLA as examples of systems that can use the tokens.

Tests in simulation and on physical robots showed better performance on dexterous manipulation tasks while maintaining computational efficiency. The work also describes the representation as extensible to new sensor types. That could make the approach useful when a manipulation system needs to handle touch inputs from more than one kind of sensor.

The LingBot-Touch implementation is available as open-source software under the Apache License 2.0.

The project adds to work on tactile sensing in robotics. Tether’s QVAC software, covered in work on processing robot sensors aboard a humanoid, processes touch, force and visual signals on the robot. LingBot-Touch instead focuses on encoding tactile inputs from different sensor types for use by multiple manipulation models.