#distillation
10 briefs tagged #distillation.
Study tests strong-to-weak transfer at inference time
A stronger model built task harnesses that lifted weaker-model scores on Theory-of-Mind benchmarks.
Paper proposes causal distillation for video generation
Context-Matched Distillation aligns teacher supervision with autoregressive video inference.
Researchers shrink visual document retrieval models
DistilVDR uses dual-student distillation to build a 524M end-to-end visual document retriever.
Paper tests strong-to-weak AI scaffolding at inference
A stronger model built harnesses that lifted weaker models on Theory-of-Mind benchmarks without training.
ByteDance trains large model to rival Anthropic
The Chinese tech giant is pre-training a model that could reach 10 trillion parameters.
OPD2 improves multilingual math reasoning in Qwen3 tests
The distillation method beat OPD in English, Korean and Japanese math reasoning experiments.
Researchers refine on-policy distillation for smaller models
New arXiv papers propose capacity- and outcome-aware ways to guide student models.
Researchers propose SKILL-KD for LLM agent skill distillation
The framework turns teacher-student trajectory gaps into tested textual skill patches.
New papers probe privileged information in OPSD
Researchers test how rubrics, anchors and problem structure affect self-distilled model training.
ReflectRL trains on failed expert reasoning traces
The paper proposes using flawed expert trajectories as reflection signals in on-policy training.