Researchers target trajectory bias in driving VLMs

An arXiv paper argues that common annotation pipelines for autonomous-driving Vision-Language-Action models can bias teacher models by exposing them to logged ground-truth future trajectories. The authors say this “trajectory anchoring bias” leads models to rationalize revealed outcomes instead of inferring decisions from scene evidence, producing less causally faithful chain-of-thought reasoning and more severe hallucinations in causally challenging scenes. They introduce AD-MCQ, a multiple-choice framing for selecting among explicit trajectory candidates, and DEFT-RLVR, which defers exposure to future trajectories during reinforcement learning with verifiable rewards.