AnyViewDex: View-Invariant Dexterous Manipulation from RGB Observations

Published September 18, 2026 · Category: Robotics

Overview

arXiv:2609.20107v1 Announce Type: new Abstract: Visuomotor policies for multi-fingered dexterous manipulation are highly sensitive to camera viewpoint shifts. To achieve view invariance, recent methods increasingly rely on explicit 3D modalities like RGB-D or point clouds, which can introduce hardware dependencies, calibration requirements, and vulnerability to sensor noise during real-world deployment. In this work, we show that view-invariant control can be achieved without explicit test-time 3D sensing by encoding geometric knowledge into the visual representation during simulation. We present AnyViewDex, an asymmetric training pipeline that combines multi-view contrastive alignment with privileged 3D geometric supervision. By regressing absolute 3D object coordinates during simulated training, this auxiliary objective provides a geometric grounding signal that mitigates the spatial collapse of the globally pooled contrastive embedding. At deployment, the policy operates zero-shot using only uncalibrated monocular RGB and proprioception. We validate this approach across both reinforcement learning and student-teacher distillation. In hardware evaluation on an xArm7 with a 16-DoF LEAP Hand, AnyViewDex reaches 76.7% grasping success across eight unseen objects and six uncalibrated viewpoints (480 trials; 2,400 across all ablation conditions), indicating that geometrically grounded monocular policies transfer zero-shot without test-time depth. Project Page: https://anyviewdex.github.io/

Source

Originally published at arxiv.org.

Related Articles

• GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning
• Time-Efficient Iterative Learning Planning for Safety-Critical Dynamic Obstacle Avoidance
• MoWAM: Explicit Future Motion Prediction for Efficient World Action Models