{\tau}: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision

Published July 28, 2026 · Category: Robotics

Overview

arXiv:2607.24485v1 Announce Type: new Abstract: Learning the informative tactile representation while effectively adapting it to pretrained Vision-Language-Action (VLA) models remains challenging at both the data and modeling levels. At the data level, limited task-specific demonstrations constrain representation quality, whereas large-scale pretraining incurs substantial costs. At the modeling level, existing methods either focus on instantaneous contact states or model temporal interaction dynamics using 6D wrench sequences, leaving high-dimensional tactile signals underexplored. To address these challenges, we present {\tau}, a touch-augmented VLA framework that learns an action-conditioned spatiotemporal tactile representation from future visual supervision inspired by the Joint-Embedding Predictive Architecture (JEPA), and fuses it with vision-language features for action generation under limited data. This supervision operates in latent space and is used only during training, adding no deployment overhead. We also introduce TacAura, a dataset of synchronized vision, proprioception, and vision-based tactile signals across four representative contact-rich manipulation tasks. Experiments show that {\tau} outperforms existing models and generalizes to unseen objects and scenes, delivering improved manipulation performance and robustness

Source

Originally published at arxiv.org.

Related Articles

• FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking
• A Visuo-Tactile Data Collection System with Haptic Feedback for Coarse-to-Fine Imitation Learning
• UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agents