VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation

Published July 3, 2026 · Category: Robotics

Overview

arXiv:2607.02503v1 Announce Type: new Abstract: Contact-rich manipulation requires policies to react to local deformation, pressure, slip, and friction, yet these cues are temporally sparse and often invisible in visual observations. Existing visual-tactile policies usually feed tactile observations directly into action prediction, but rarely model tactile deformation dynamics during action generation. In this paper, we introduce VT-WAM, a Visual-Tactile World Action Model that jointly learns future visual prediction, tactile deformation prediction, and action prediction within a unified flow matching framework. In particular, VT-WAM introduces (1) Asymmetric Mixture-of-Transformers (MoT) attention to bridge a first-frame visual anchor with temporal tactile dynamics, and (2) contact-gated Action-Visual-Tactile Attention Guidance (AVTAG) to encourage action queries to rely on tactile evidence during contact phases. Across six real-world contact-rich manipulation tasks, VT-WAM achieves a 71.67% average success rate, outperforming Fast-WAM by 26.67% and OmniVTLA by 35.84%. Ablations demonstrate that modeling tactile deformation dynamics and guiding contact-phase tactile attention are both important for contact-rich tasks. Project website: https://vt-wam.github.io/.

Source

Originally published at arxiv.org.

Related Articles

• Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group
• One Demonstration Is Enough for Real-World Robotic Reinforcement Learning
• Human Supervisor Workload Prediction: Lag Horizon Selection