NVIDIA packs 74B parameters into a 24GB file for local agents

NVIDIA released Nemotron 3.5 Bolt 74B-A9B 24GB on Thursday, Oct. 15, describing it in the company's own wording as "a new LLM purpose-built for running local agents on RTX Spark," the laptop platform that goes on sale the next day. The name carries the pitch: 74 billion total parameters, 9 billion active per token, packed into a 24GB footprint. It is the load-bearing piece of NVIDIA's month-long argument that a private agent can run on the machine in front of you — and on the evidence published so far, it is also a launch where the checkable part is the footprint and almost everything else is the vendor's word.

What the 24GB figure conceals

NVIDIA's blog post calls Bolt an LLM. The model card underneath it is more specific and slightly different: a vision-language model that accepts text, images and video and returns text, built on a Mamba-2 and Transformer hybrid LatentMoE backbone with a C-RADIO v4-H vision encoder. The blog says the 24GB target is reached through 2-bit quantization with quantization-aware training; the card names the precision IQ2. The arithmetic behind the headline is the familiar mixture-of-experts trade: active parameters set compute per token, total parameters set the memory bill, so 9B of the 74B are read on each step while all of the weights stay resident. The card lists a context window of up to 256K tokens.

The scores exist, and NVIDIA graded them

Unaudited vendor numbers, below the line, with no third party having rerun them. NVIDIA's blog published no benchmark table at all; the model card does, pitting Bolt against two competing small MoEs and NVIDIA's own earlier Lightning 30B-A3B.

Benchmark	Bolt 74B-A9B (IQ2)	Qwen3.6-35B-A3B (BF16)
GPQA Diamond	86.43	83.40
Terminal-Bench 2.1	50.98	44.38
AA-LCR	64.06	61.06
AA-Omniscience accuracy	16.90	19.47

The final row is the interesting one. Bolt leads or ties across most of the table, but on AA-Omniscience accuracy it finishes behind both Qwen3.6-35B-A3B and the smaller Gemma4 26B-A4B, and its non-hallucination score falls below its own Lightning sibling. A footnote concedes that a rival's published MCP-Atlas number could not be reproduced at the same value. Rival scores inside a vendor table are vendor-measured too, which makes the whole grid a claim rather than a measurement.

The laptop is the product the model serves

Bolt arrives a day before RTX Spark laptops, which NVIDIA and Microsoft said would be available on Friday, Oct. 16. The platform is specified at up to 128GB of unified memory and 1 petaflop of AI compute, and NVIDIA frames the model as a reason for it to exist. Agent apps including GitHub Copilot, Hermes, OpenClaw, Perplexity, Pi and WorkBuddy ship streamlined setup on the hardware; NVIDIA OpenShell and Microsoft Execution Containers add operating-system-level sandboxing; and NVIDIA Switchyard, an open-source router, moves queries between local and cloud models. NVIDIA PAIR left beta and added llama.cpp support. The same post asserts that local LLM downloads are already 5x higher than all of last year, and that models such as Qwen 3.8-Flash-Next are catching up to frontier cloud models released a few months ago, both offered as NVIDIA's framing without a dataset behind them.

Tooling was ready before the weights

Support landed ahead of the release rather than trailing it. A llama.cpp pull request, #30520, adding Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B support, was merged on Oct. 15 — the release day itself. The card lists llama.cpp, vLLM, Ollama, Unsloth and LM Studio as launch paths and specifies a LiLiCorr drafter with seven speculative tokens in the companion vLLM recipes. Tooling readiness is the one claim in this launch a reader can independently confirm, and it checks out.

Open weights under a custom licence

NVIDIA says Bolt is fully open source, including training data, weights and recipes. The repository tells a narrower story: the weight files carry license "other", named OpenMDW-1.1, a custom agreement rather than a standard open-source licence, so open-weight is the accurate description and open source is a claim to test against the terms. The family tree adds a detail worth reporting without resolving: the BF16 checkpoint's declared base model is NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16, with the relation marked as a fine-tune, so the 74B model is presented as a descendant of a larger sibling. The card's comparison table also names a different model, Nemotron 3.5 Lightning 30B-A3B — a separate release, not another name for this one.

What the release thread noticed

On r/LocalLLaMA the weights carried the New Model flair in a thread whose title used the community's own ordering, Nemotron Bolt 3.5 74B A9B. The comments read cooler than the press framing. One wrote that "Fast is fun but it doesn't mean the quality of the answer you'll get is any good," doubting it would match Qwen out of the gate; another called the earlier Nemotron Ultra among the worst functional models they had tried; a third read the architecture as aimed at throughput rather than top-tier quality. One reply defended the quantization-aware training as the reason it "still has a shot at being pretty good." Hugging Face's counters were modest at the time of writing: 58 downloads and 67 likes on the packed repository, 119 downloads on the GGUF copy.

What would settle it

Two things are missing. The first is an independent evaluation: a third party running Bolt on named hardware, at the stated IQ2 precision, with memory actually measured, since every quality figure on the table is NVIDIA's own. The second is a comparison at the headline claim itself — a 24GB footprint against the models people already run inside that budget, on a multi-step agent job rather than a single-turn benchmark. Until someone publishes that, the defensible part of this release is what shipped: a 74B sparse model in a 24GB file, under a custom licence, with tooling that was waiting for it.