Observing and Controlling Features in Vision-Language-Action Models
arXiv:2603.05487v2 Announce Type: replace Abstract: Vision-Language-Action models (VLAs) have shown remarkable progress towards embodied intelligence. While their architecture partially resembles that of Large Language Models (LLMs), VLAs exhibit higher complexity due to their multi-modal inputs/outputs and often hybrid nature of transformer and diffusion heads. This is part of the reason why insights from mechanistic interpretability in LLMs, which explain how the internal model representation
Overview
arXiv:2603.05487v2 Announce Type: replace Abstract: Vision-Language-Action models (VLAs) have shown remarkable progress towards embodied intelligence. While their architecture partially resembles that of Large Language Models (LLMs), VLAs exhibit higher complexity due to their multi-modal inputs/outputs and often hybrid nature of transformer and diffusion heads. This is part of the reason why insights from mechanistic interpretability in LLMs, which explain how the internal model representations relate to their output behavior, do not trivially transfer to VLA counterparts. In this work, we investigate whether VLA internal representations support lightweight behavioral steering without retraining. Across four frontier VLA models, linear \emph{observers} recover state- and action-relevant information in both autoregressive and transformer--flow-matching architectures, and provide robust directions to causally alter VLA outputs. Building on this, we introduce a \emph{controller} that minimally modifies representations to place observer predictions within prescribed target intervals. Closed-loop experiments with $\pi_{0.5}$ in the LIBERO simulator and on DROID hardware demonstrate improved constraint satisfaction while retaining task performance, with only approximately $1\%$ inference overhead. Together, these experiments show that lightweight linear interventions can reliably steer VLA behavior while preserving closed-loop capabilities, enabling alignment with user preferences and task requirements without fine-tuning.
Source
Originally published at arxiv.org.
Related Articles
Source: https://arxiv.org/abs/2603.05487