Nemotron 3.5 Embed and Rerank: Models, Benchmarks and RAG Deployment

NVIDIA introduced Nemotron 3.5 Embed and Rerank in an October 21, 2026 announcement, following model releases dated October 20 on Hugging Face. The models retrieve and rank evidence from text and document images, including charts and tables. They target a practical agent problem: finding better evidence before asking a larger model to reason over it.

This release is distinct from document parsing. A parser turns a page into structured content. An embedding model makes content searchable, while a reranker examines the retrieved candidates more closely.

Which Nemotron 3.5 retrieval models are available?

The official announcement introduces three embedding variants and one reranker. Model cards identify the released weights as available for commercial and non-commercial use under OpenMDW-1.1.

Model	Role	Standard output
Embed 8B BF16	Text and image retrieval	4,096-dimensional vector
Embed 1B BF16	Smaller text and image retriever	2,048-dimensional vector
Embed 1B NVFP4	Quantized retriever for supported hardware	2,048-dimensional vector
Rerank 8B BF16	Reorders retrieved query-document pairs	Relevance score

The names are size labels rather than exact parameter counts. The cards report 8.38 billion parameters for Embed 8B and Rerank 8B, and 1.55 billion for the 1B embedding models.

What changes from Nemotron 3 Embed?

The 8B embedding model card says its language component remains frozen while image capabilities are added. It shares the corresponding Nemotron 3 model's text embedding space and produces identical text embeddings.

The 1B card describes the same text-space continuity for its matching predecessor. Both BF16 models accept text queries and documents containing text, images or both, with a 32,768-token maximum context.

For teams already using those specific predecessors, this offers a migration path that does not automatically require rebuilding every text vector. It does not mean arbitrary existing indexes are compatible. Dimensions, preprocessing and the specific predecessor still matter.

The useful first experiment would preserve the existing text corpus and add a small collection of documents where layout or images carry essential information. That separates the value of visual retrieval from changes caused by rebuilding the whole pipeline.

Why use a separate reranker?

The Rerank model jointly evaluates a text query and each candidate document, which may contain text, a page image or both. Its output is a raw relevance logit, with larger values indicating stronger relevance.

That is different from searching a database of precomputed vectors. Reranking performs more work on a smaller candidate set after initial retrieval.

The extra stage has a cost, so the right question is whether better evidence saves more work later. An agent that receives a relevant table immediately may avoid another search and another long model call. An agent working on simple text questions might gain less.

The score also should not be presented to users as a probability that an answer is correct. It ranks relevance between a query and a document; answer correctness remains a separate evaluation.

What do the benchmarks show?

NVIDIA reports 61.17 nDCG@10 for Embed 8B BF16 and 58.38 for Embed 1B BF16 on ViDoRe v3.1. This metric rewards relevant documents appearing near the top of search results.

In NVIDIA's agent experiments, adding its reranker reduced agent inference tokens per query by approximately 14% with Embed 8B and 13% with Embed 1B, while improving retrieval accuracy.

Those are vendor-reported results under a fixed experimental setup. Fewer agent tokens are encouraging, but they are not automatically an equal reduction in total serving cost. The embedding and reranking stages consume resources too.

Deployment and the NVFP4 tradeoff

The NVFP4 model card specifies vLLM and hardware with NVFP4 support, such as NVIDIA Blackwell GPUs. It recommends the BF16 version for Transformers or Sentence Transformers.

NVIDIA says the BF16 and NVFP4 1B variants generally share an embedding space, while recommending validation on representative data before switching. That is a useful caution: a smaller numeric format can change behavior enough to matter on a particular collection.

For an initial comparison, hold the agent model and document set fixed. Measure retrieval quality, end-to-end latency and the total cost of a completed task. Include difficult pages where the answer depends on a chart or table rather than ordinary prose.

The strongest reason to adopt these models would be a measurable improvement in that complete workflow. A better retrieval score becomes useful when the agent reaches the right evidence with less wasted work.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams explain technical launches and developer workflows in content that founders and engineers actually read and share.

Book a 30-minute call.