Latest in AI
SlideOn and mSlideOn: LightOn's Sparse Retrieval Models Explained
LightOn's new checkpoints make retrieval sparsity adjustable, giving search teams another way to balance relevance against index size.
LightOn announced SlideOn and mSlideOn on October 19, releasing English and multilingual sparse retrieval checkpoints with 8,192-token input windows. The models represent text through a learned set of active features. Teams can prune those representations to change the retrieval footprint without retraining the model.
The release announcement includes trained retrieval models, sparse-autoencoder pretraining checkpoints, and training and evaluation resources. This is a self-hosted retrieval release. It does not announce a hosted API or per-token service price.
SlideOn and mSlideOn specifications
| Model | Main scope | Parameters | Maximum input | Learned feature space |
|---|---|---|---|---|
| SlideOn | English retrieval | 174 million | 8,192 tokens | 32,768 features |
| mSlideOn | Multilingual retrieval | 339 million | 8,192 tokens | 32,768 features |
The input limit includes prompts and special tokens. The SlideOn model card and mSlideOn model card provide usage instructions and benchmark results. The multilingual model's training coverage includes nine natural languages plus code; code should not be counted as a tenth human language.
LightOn describes the release as permissively licensed. The code repository has an Apache 2.0 license, but the inspected model-card metadata does not explicitly declare a weight license. Check the applicable model artifacts and terms before assuming that the code license also governs the weights.
How sparse retrieval works here
A dense retriever represents a document with a vector whose dimensions generally remain populated. These models instead produce sparse vectors over a learned dictionary of 32,768 features. Most dimensions can be inactive, and retrieval uses a dot product between the query and document representations.
The features are learned latent dimensions, not ordinary words with a direct vocabulary label. An inverted index can use shared active dimensions to find candidate matches, but this is not simply keyword search with a larger list of words.
The architecture uses a sparse autoencoder to learn the feature dictionary, followed by retrieval training. It pools token-level representations into a document or query vector. Matryoshka-style training supports pruning those final representations, allowing a team to change the number of retained features without training a new checkpoint each time.
The practical choice is how much representation to keep for a particular corpus. Fewer active features can reduce the index footprint and overlap work, but may remove information needed to retrieve the right document.
What the benchmarks show
LightOn reports a BEIR 15-dataset nDCG@10 score of 56.68 for SlideOn, compared with 56.20 for DenseOn and 57.22 for LateOn. That puts the English sparse model slightly above the dense comparison on this aggregate, while below the late-interaction model.
For mSlideOn, the reported MIRACL score is 62.78 versus 58.02 for mDenseOn. The long-document MLDR result is 75.79 versus 65.62. However, mSlideOn's English BEIR score is 56.00, below mDenseOn's 56.70. The results do not support a claim that sparse retrieval is uniformly better.
These are vendor-reported retrieval evaluations, not independent reproduction. Aggregate benchmark gains also cannot tell a team whether its own documents, languages or queries will improve. A support knowledge base with short repetitive entries can behave differently from a long-document research collection.
LightOn also reports RetrievalFLOPS, a proxy based on overlap between active features. It is not measured end-to-end latency or the encoder's arithmetic workload. Treating that number as a guaranteed search-speed or cost improvement would skip the effects of indexing, hardware and implementation.
How to evaluate the models in an existing search system
Start with a fixed set of real queries and judged relevant documents. Encode both sides with the appropriate query and document methods, then compare relevance at several pruning settings. The model-card interface uses SparseEncoder, with encode_query and encode_document, and returns sparse COO tensors.
The multilingual card gives an example retaining 128 query features and 1,024 document features. Those final-vector settings are separate from token-level sparsity. They are a starting configuration, not a universal recommendation for every corpus.
The cards also require custom code through trust_remote_code=True. Review and pin that code before using it in your own environment. This article has not executed the models or benchmarked a retrieval backend.
An existing dense index will need a suitable replacement or parallel sparse index, along with document encoding. Budget for that migration rather than assuming a model-name change is sufficient. Measure encoding time, index size, search latency and relevance separately so a faster search stage does not hide a more expensive ingestion process.
The release is most useful for teams willing to tune that balance. It offers a way to adjust representation size after training, but the right setting still comes from the queries the search system must answer.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help technical teams explain retrieval systems, model releases and engineering tradeoffs in content that developers and founders can understand and share.