Researchers target VLM gaps in spatial reasoning
COVT adds continuous visual tokens as GST-Bench measures global spatial awareness failures.
Why it matters
The work highlights a persistent weakness in VLMs: models can handle language reasoning and some local visual tasks but struggle to build globally consistent spatial representations. That gap matters for embodied agents and video-understanding systems that must reason across long-horizon visual streams.
The key points
- 1.COVT uses roughly 20 continuous visual tokens for dense perceptual cues.
- 2.GST-Bench tests spatial inference from unseen viewpoints in video.
- 3.Top zero-shot VLMs remain far below human global spatial awareness scores.
Researchers introduced Chain-of-Visual-Thought, a framework that trains vision-language models to reason with compact continuous visual tokens carrying cues such as 2D appearance, 3D geometry, layout and edges. Separately, GST-Bench evaluates global spatial intelligence in video using human-verified questions from 6,790 minutes of synthetic video and reports that the strongest zero-shot VLM scored 42.68 versus a human score of 79.08.
⚡ Try this today
Use GST-Bench-style evaluations before relying on VLMs for navigation, robotics or global scene-understanding tasks.
Sources & original reporting
This brief summarizes and links to reporting from the publishers below.
- arXiv cs.AIChain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual TokensAug 6, 12:00 PM↗
- arXiv cs.LGChain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual TokensAug 6, 12:00 PM↗
- HF Daily PapersGST-Bench: Can VLMs Develop Global Spatial Awareness from Video?Aug 6, 4:00 AM↗
Enjoyed this brief? Get the next one in your inbox.
More in Research
New studies probe spatial reasoning in vision models
SpaRRTa, SMA and PinpointQA target gaps in model spatial understanding for embodied AI.
CW-BASS v2 targets pseudo-label filtering with DINOv2 teachers
The arXiv paper proposes a saturation-aware method for semi-supervised semantic segmentation.
Study tracks ChatGPT Enterprise use across organizations
The paper links account records to roles, tasks and public-company data through March 2026.