DenseOn Unsupervised
lightonai/DenseOn-unsupervised
published Mar 2026 · updated Apr 2026
DenseOn Unsupervised is an embed model that provides a strong intermediate dense checkpoint from the first stage of the DenseOn pipeline, using unsupervised contrastive pre-training on filtered query-document pairs.
specs
| Task | Embeddings / Dense Retrieval |
| Architecture | ModernBERT-base with CLS token pooling |
| Parameters | 149M |
| License | Apache 2.0 |
best for
- ·Starting point for supervised fine-tuning on domain-specific data
- ·Research ablations on hard-negative mining strategies
- ·Knowledge distillation experiments from a strong unaligned base
FAQ
It is intended as a starting point for supervised fine-tuning, knowledge distillation, or research ablations, not for direct production retrieval.
DenseOn Unsupervised (49.05 BEIR NDCG@10) is the pre-training-only checkpoint; the full DenseOn (56.20) adds supervised fine-tuning with mined hard negatives and is recommended for production.
It uses a ModernBERT-base backbone with 149M parameters, 768-dimensional embeddings, and CLS token pooling with a 512 token limit.
Apache 2.0.
Use the gigarouter OpenAI-compatible endpoint with your API key, specifying the model ID lightonai/DenseOn-unsupervised.
We're benchmarking and onboarding DenseOn Unsupervised as a hosted, OpenAI-compatible API. Sign in for free credit and be ready when it lands, or tell us you want it and we'll prioritize it.