Researchers release UEmbed multimodal embeddings
UEmbed generates sparse lexical and dense representations in one decoder-only forward pass.
Why it matters
UEmbed points to a simpler architecture for combining learned sparse retrieval and dense multimodal embeddings. That could matter for search and retrieval-augmented generation systems that need both lexical precision and semantic matching.
The key points
- 1.Produces sparse and dense embeddings in one causal pass.
- 2.Released at 2B, 4B, and 9B scales.
- 3.UEmbed-9B reports 71.8 dense on MMEB-v2.
Researchers introduced UEmbed, a decoder-only multimodal embedding model that produces both sparse lexical and dense representations in a single causal forward pass. The model appends learnable special tokens to the input, partitions the vocabulary into disjoint subsets, and uses each token's hidden state to predict sparse weights. The team released 2B, 4B, and 9B versions trained on public data, with UEmbed-9B scoring 71.8 dense and 71.0 sparse on MMEB-v2.
⚡ Try this today
Read the paper before choosing separate sparse and dense multimodal embedding stacks for new retrieval systems.
Sources & original reporting
This brief summarizes and links to reporting from the publishers below.
Enjoyed this brief? Get the next one in your inbox.
More in Research
[Paper] Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning
We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iter
Google applies homomorphic encryption to private AI
Google says encrypted processing can help make private AI more practical.
Google advances private AI with homomorphic encryption
Google says it is making private AI more practical using homomorphic encryption.