NVIDIA Nemotron Bolt E4B: What the 4GB Local-Agent Model Can Run

NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23 for local assistants, interactive characters and robots. Its launch demonstration pairs Bolt for conversation with Cosmos for vision, running both locally on a Jetson Orin Nano 8GB.

The useful question is how much room an application needs around the language model. A voice assistant has more to do than produce text: it must listen, decide whether to act, wait for tools and speak. Each part competes for the same hardware budget.

What Bolt E4B includes

The model card identifies a Gemma-4-E4B architecture with 8 billion total parameters and 4 billion effective parameters. This checkpoint uses Q4_K_M quantization, accepts text and produces text, with support for up to 64K context. It is aimed at conversation and tool use, rather than being a standalone vision or speech model.

Developers can switch reasoning on or off and enable a conversational mode through the chat template. The card also provides a llama.cpp serving example and an optional DFlash draft model. Those controls give a developer several configurations to evaluate instead of one fixed chat behavior.

For example, I would test a short device command separately from an ambiguous planning request. Turning on more reasoning for every interaction could make the common path unnecessarily slow. Turning it off everywhere could leave harder requests without enough deliberation. The right policy should follow measured task performance.

Why 4GB is a workload target

NVIDIA's technical launch post reports 3.5GB of VRAM and a 548-millisecond time to first token on an RTX 5070 for a 3K-input, 50-output-token workload without cached context. The same workload on Jetson Orin Nano uses 4.2GB and takes 5,785 milliseconds to produce its first token.

With 32K cached tokens, 2K new input tokens and 256 output tokens, reported memory rises to 4.3GB on the RTX 5070 and 4.9GB on the Orin Nano. These measurements use Q4_K_M, llama.cpp CUDA and batch size one. The name therefore should not be read as a universal four-gigabyte ceiling.

For a product team, that makes first-token delay worth testing before throughput. A device can generate the rest of an answer at a reasonable pace and still feel unresponsive if the user waits too long for it to begin. Measure the full interaction, including microphone processing and speech output, on the intended device.

Read the benchmark gains as NVIDIA's results

NVIDIA reports roughly 40% higher average benchmark scores than Nemotron 3 Nano 4B, with up to 80% fewer reasoning output tokens. Those are vendor-reported results, not a promise of equivalent gains in every application.

I would build a small evaluation around missed tool calls, unnecessary tool calls, contradictory instructions and interrupted conversations. Include cases where the correct behavior is to ask a question or do nothing. A conversational agent that acts confidently on the wrong request can be less useful than one that answers more slowly.

The weights are available through Hugging Face under OpenMDW-1.1, as specified in the model card. Running them locally still requires hardware, integration and maintenance. The practical next step is a representative device test, not an assumption that an open download removes deployment costs.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI founders turn technical launches and product updates into clear stories that developers and founders read and share.

Book a 30-minute call.