Maxwell-V1-Flash bounds attention at 4,096 tokens. Its KV cache still fills a 24 GB card
Maxwell-V1-Flash bounds attention at 4,096 tokens per KV group but keeps the full KV history, so the 35B/3B MoE is cheap in FLOPs and costly in memory.

Maxwell-V1-Flash is a 35-billion-parameter mixture of experts that activates 3 billion parameters per token, both figures excluding embeddings, released under Apache-2.0 with a 256K context window. The model card hangs the launch on one architectural claim: Maxwell Block Sparse Attention, which caps main attention at 4,096 tokens per KV group and, on the vendor's own kernel measurement, runs 6.7 times faster than dense grouped-query attention at full context.
That is a compute number, and a real one. It is also the only number the launch states in prose, because the benchmark results arrive as an image rather than a table, and the card keeps what sparse attention usually lets you throw away: the full-history KV cache. Every token inside a 256K window keeps its keys and values resident. The model is cheap to think with and expensive to hold, which is close to the inverse of what a 35B/3B label suggests to somebody shopping for a 24 GB card.
What the card commits to
The card is unusually explicit about the architecture, which means the sparse-attention claim can be checked on paper even though the model's quality cannot be.
| Property | Value |
|---|---|
| Total / active parameters | 35B / 3B, excluding embeddings |
| Context | 256K tokens, text only |
| Blocks | 45, of which 3 dense and 42 MoE |
| Experts | 256 routed, top-12, plus 2 shared |
| Hidden size / vocabulary | 2,048 / 200,704 |
| Attention heads | 32 query, 4 KV, RoPE base 10,000,000 |
| Pre-training | ~26T tokens, Muon, block-sparse enters at 128K mid-training |
| Licence | Apache-2.0 |
Two entries do most of the work below: four KV heads against 32 query heads, and a hidden size of 2,048. Together they fix the size of the cache that the sparse mask never touches.
The bound is exact, and it is 32 blocks
The pattern is not vague about what it keeps. Each KV group holds fixed coverage of eight prefix blocks and eight local blocks of 128 tokens, plus up to 16 remote blocks chosen by a learned indexer. Eight plus eight plus sixteen is 32 blocks, and 32 times 128 is 4,096 tokens, exactly the ceiling in the card. What the indexer buys is not more attention but better placement of a fixed budget: 16 distant blocks can sit on a retrieved document, a function signature or a system prompt, wherever the indexer estimates they matter.
That is a genuine cut in work. At 256K, attention over reachable history stops growing with the sequence, which is the regime where a 6.7x kernel advantage becomes plausible rather than promotional. The measurement is the vendor's, and nobody has posted a second one.
The cache is where the bill lands
Sparse selection reduces what the model reads on each step. It does not reduce what the runtime has to keep, because the card says the full-history cache stays available, and the indexer can only choose from blocks still in memory. Cache size therefore follows head geometry, not the mask: 45 blocks, four KV heads, a head dimension of 64 implied by 2,048 split across 32 query heads, and two tensors per layer in two bytes each. That is roughly 45 KB per token, or about 12 GB at 262,144 tokens, before allocator overhead and before a single weight loads.
The arithmetic that circulated in the thread was harsher. u/Gohab2001 wrote: "kV cache seems 4.5x Qwen 3.5 35B. Nearly 24GB @ 256k context". Both readings describe the same wall from different sides, and the conclusion does not depend on which is right: the cache sets the floor on the card you need. Meanwhile 35 billion parameters have to live somewhere even when 3 billion are awake, which is why offload behaviour matters as much as the sparse mask. Work like llama.cpp's MoE expert cache work decides whether the sleeping parameters can sit in system memory without turning every token into a PCIe stall.
A model that appeared in two pieces
The artefact trail is short enough to list. The Hugging Face repository was created on 13 October 2026 and last modified on 14 November; by publication it showed 287 downloads and 148 likes. The GitHub repository contains exactly one commit, "Initial commit", timestamped 2026-11-13T13:10:31Z and unsigned, and it is where the technical-report PDF lives. The project site dates that report 30 October 2026 and titles it "Maxwell-V1.0-Flash: Strong Reasoning and Agentic Search with 3B Active Parameters", advertising the weights under Apache 2.0.
So the paper predates the repository by two weeks, and the repository predates the thread that made the model visible by four days. Writing the report first is normal practice and proves nothing either way. What it does mean is that the document was not revised against a tagged release, and that the single commit is unsigned, so no verifiable identity sits behind the code.
The Qwen question, answered
The loudest doubt in the thread's first hours was lineage. u/Dabalam asked it directly: "Can some one tell me if this is a Qwen fine tune in a trenchcoat or actually a new model?" u/JaredsBored did the checking and answered: "Different architecture, different vocab size (200k vs 151k), and new attention type vs qwen 3.5/3.6 35b. Definitely not a fine tune but I had to check when I saw the param size lol".
On the published evidence the second answer holds. A vocabulary of 200,704 tokens is not Qwen's 151k, and a learned-indexer block-sparse attention is a different mechanism rather than a re-parameterisation of grouped-query attention. A fine-tune inherits a tokenizer and an attention stack; this release changes both, and adds a tokenizer tuned for low-resource Southeast Asian scripts.
What is still missing
Artificial Analysis has no page for the model; the URL returns 404. There is no independent score, no third-party throughput curve, and no cost figure from anyone but the vendor. The only external activity so far is infrastructural: community GGUF quants appeared on 17 November at 08:10 UTC, the same day as the thread, which says more about demand than about quality. One early reply pointed at the gap that follows. u/wombweed wrote: "Sadly I don't see support in llamacpp which makes me feel like cpu expert offloading won't be the fastest thing".
The honest summary is narrow. Maxwell-V1-Flash is a real release of real weights under a permissive licence, with an attention design that is genuinely different from the models it will be compared against, and a memory profile that punishes the card its parameter count advertises. What is absent is the thing a launch usually leads with: a score. Until somebody independent runs it, 6.7x is a kernel benchmark, 4,096 is an architectural bound, and 24 GB is arithmetic done by a stranger in a comment thread. All three are checkable. None of them is a claim about what the model can do.


