Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition

Published September 15, 2026 · Category: Robotics

Overview

arXiv:2609.14783v1 Announce Type: new Abstract: Robots need touch to manipulate objects safely and reliably, as many properties, such as softness, texture, and contact stability, are hard to infer from vision alone. However, vision-based tactile sensors yield different observations of the same material due to variations in optics, elastomer properties, and illumination, leading to poor generalization when trained on a single or multiple sensors. We propose a language-guided distillation framework for learning sensor-robust tactile representations. Language encodes high-level semantic properties of touch (e.g., rough, soft, slippery) that remain invariant across sensing hardware, providing a natural sensor-agnostic supervisory signal. We construct a 39K-sample touch-language dataset with human-annotated material labels and train a tactile encoder to align sensor-specific tactile images with language embeddings in a shared semantic space. We evaluate our approach for few-shot learning and cross-sensor transfer and benchmark it on six existing tactile datasets. Our method achieves 95% accuracy in the 100-shot setting, improves cross-sensor transfer by an average of 13.3% accuracy, and yields up to 19% accuracy gains across six existing tactile datasets. These results demonstrate that language-guided distillation enables scalable and hardware-agnostic tactile representation learning. Code and dataset are available at https://mashood3624.github.io/Language_Tactile/

Source

Originally published at arxiv.org.

Related Articles

• Conflict-Predictive Variable Horizons in Multi-Drone Distributed Model Predictive Control
• IMM-based Multiple Object Tracking using a State Prediction Neural Network
• Constraint-Grounded Reinforcement Learning for Variable Impedance Control in Contact-Rich Robotic Insertion