ARCHead targets LLM output-head compression
The method compresses LM-head storage without keeping a dense BF16 head, according to a new arXiv paper.
Why it matters
Output heads are often left in BF16 or FP16 even when transformer blocks are quantized, leaving a storage gap in practical deployments. ARCHead could make weight-only quantized LLMs smaller by addressing that remaining projection layer.
The key points
- 1.Compresses LM-head storage by 3.7-3.9x.
- 2.Uses 25.6% of BF16 head storage on Qwen3-8B-Base.
- 3.Adds less than 2% throughput change in reported tests.
Researchers introduced ARCHead, a packed compressor for large language model output heads that combines a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric. The paper says ARCHead reduces persistent LM-head storage by 3.7-3.9x and, on Qwen3-8B-Base, uses 25.6% of BF16 head storage while reaching 1.007 relative perplexity. Replacing the BF16 head left by AWQ or bitsandbytes added 0.006-0.007 cross-entropy with less than 2% throughput change in the authors' measurements.
⚡ Try this today
If you deploy AWQ or bitsandbytes-quantized models, review the ARCHead code and test output-head compression on your target model before relying on BF16 heads.
Sources & original reporting
This brief summarizes and links to reporting from the publishers below.
- arXiv cs.CLARCHead: Activation-Metric Residual Correction for Large Language Model Output HeadsAug 5, 12:00 PM↗
- arXiv cs.LGARCHead: Activation-Metric Residual Correction for Large Language Model Output HeadsAug 5, 12:00 PM↗
- HF Daily PapersARCHead: Activation-Metric Residual Correction for Large Language Model Output HeadsAug 3, 4:00 AM↗
Enjoyed this brief? Get the next one in your inbox.
More in Research
Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency
arXiv:2608.
Improving the matrix multiplication exponent with modern optimization and AlphaEvolve
arXiv:2608.
[Paper] Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning
We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iter