New papers test self-distillation for agentic RL

A set of new arXiv papers examines self-distillation methods for training language-model agents with sparse rewards. AgentOPSD proposes a critic-free recursive method that turns token-level teacher-student log-probability gaps into turn-level credit signals, evaluated on ALFWorld, WebShop and Search-QA with Qwen2.5 3B and 7B models. OCSD proposes contrasting full and observation-ablated replay views to reduce scaffold-related confounding when future observations are used as privileged information. A separate paper argues that privileged-information self-distillation can reproduce gains in easy settings but fails or degrades validation accuracy on harder question-answering, math, coding and agentic tool-use tasks when used as the sole objective.