Temporal Self-Imitation Learning

Published June 19, 2026 · Category: Robotics

Overview

arXiv:2606.19752v1 Announce Type: new Abstract: Long-horizon robot manipulation policies trained with reward shaping can still exploit dense rewards through inefficient interaction, while rare efficient behaviors may be forgotten during training. We argue that temporal efficiency itself provides a powerful and underutilized source of self-supervision for reinforcement learning. We introduce Temporal Self-Imitation Learning (TSIL), a reinforcement learning framework that mines temporally efficient successful trajectories generated during learning and converts them into reusable supervision for future policy improvement. TSIL progressively refines learning using configuration-conditioned adaptive temporal targets derived from fast successful trajectories, while preserving and replaying efficient behaviors through efficiency-weighted self-imitation learning. Across 15 distinct long-horizon manipulation tasks, TSIL consistently improves learning efficiency, task-completion efficiency, revisitation of fast successful behaviors, and robustness to unstable training conditions. More broadly, our results suggest that the temporal structure of successful behavior itself provides a scalable self-supervisory signal for reinforcement learning beyond manually engineered reward shaping alone.

Source

Originally published at arxiv.org.

Related Articles

• pdSTL: Probabilistic Differentiable Signal Temporal Logic for Stochastic Systems
• Fail-RAG : A Retrieval Augmented Generation Informed Framework for Robot Failure Identification
• CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotion