All News
sensenova-looplooped-transformermodel-releasehugging-facelocal-inferencecontext-window

SenseNova-Loop-9B runs 56 layers on the weights of 32, and bills you per pass

SenseNova-Loop-9B stores 32 layers, executes 56 per token, and lands near Qwen3.8-27B on two vendor benchmarks while sitting 3.2 points lower on a third.

Vlad MakarovVlad Makarovreviewed and published
5 min read
SenseNova-Loop-9B runs 56 layers on the weights of 32, and bills you per pass

SenseNova published SenseNova-Loop-9B on 17 November: weights on Hugging Face, code on GitHub, Apache 2.0, images in and text out. The claim worth examining is a trade rather than a score. The checkpoint stores 32 decoder layers and executes 56 per forward pass — layers 0 to 3 once, 4 to 27 twice, 28 to 31 once — so the middle of the network is reused, not duplicated. Nothing extra has to fit in memory. Everything costs more time.

The card states both halves: approximately 9B total parameters, approximately 14.6B inference-equivalent — what you download, and what you pay for.

Thirty-two sets of weights, fifty-six layers of work

Looping buys depth without buying parameters, which is why it recurs in research and rarely ships at this size. Layers 4 to 27 run twice, the first pass feeding the second, so the middle of the model is a function applied to its own output — the card writes it B(B(h)). Two residual streams run through that block while a learned map decides how each layer reads across them and writes back, Sinkhorn-normalised over 20 iterations to keep the streams distinct.

PropertySenseNova-Loop-9B
Distinct layers / executions32 / 56
Looped range / passes[4, 28) / 2, shared weights
Total / inference-equivalent parameters~9B / ~14.6B
Context512K tokens, trained on text, images, code, tool traces

The attention detail is where looped models quietly break. Each group of four layers holds three linear-attention layers and one full-attention layer, and the loop boundary sits on a complete group, so both passes see the same pattern. A cut through a group would still run, and would be a different model.

The comparison the card leads with

The vendor's framing is one sentence: in agentic and data analysis work the model is "competitive with Qwen3.8-27B". Here are the figures behind it, with the Preview column the card also publishes.

BenchmarkLoop-9BPreviewQwen3.8-27B
NL2Repo41.36chart only44.60
AIDABench65.31—65.01
AutomationBench36.6714.3336.67
Terminal-Bench 2.159.5540.45—
GAIA69.9037.86—
BIRD72.3471.38—

As a contest between 9B and 27B it is one narrow win, one tie and one loss: 65.31 against 65.01 on AIDABench, an exact match on AutomationBench, and 41.36 against 44.60 on NL2Repo — a gap of 3.24 points. That is a good result for a model storing a third of the weights. It is not "9B beats 27B", and the card does not claim it.

The Preview deltas are easier to produce: they measure the release against the vendor's own earlier checkpoint. One figure circulating with them, a Preview NL2Repo score of 19.91, exists only inside the chart image; the prose names AutomationBench, Terminal-Bench 2.1 and GAIA. The comparison set drew the obvious objection in the thread — why measure a 9B against a 3B and a 40B coder?

Depth is a dial the vendor set at two

The most useful part of the card is the loop-scaling experiment. With training data fixed and only loop count varying on a 9B backbone, quality rises across four tasks and peaks at three or four passes: AIDABench-QA up 6.42% and Workbuddy-office 16.93% relative to a single pass at three loops, with LiveCodeBench V6 and AIDABench visualisation best at four, up 8.10% and 8.55%.

Then the curve flattens. The release runs two loops, and going from two to four buys 1.04 percentage points on LiveCodeBench V6 and 1.35 on AIDABench visualisation — two extra passes through a third of the network, for about a point. The vendor gives the reason plainly, two loops "to balance inference efficiency", and leaves deeper looping open, with dynamic loops that spend passes according to task difficulty as future work.

That is a latency decision wearing the costume of a quality decision. How many passes you can afford is set by your time budget, not by what the benchmark rewards.

Where the fifteen gigabytes come from

Memory is the second place the loop pays off, and where the launch figure is most often misread. At 4-bit the weights take roughly 4.9 GB; everything else in the deployment table is context.

Context8-bit KV cache4-bit KV cache
64K~10 GiB~9 GiB
128K~12 GiB~10 GiB
256K~15 GiB~12 GiB

The 3 GiB gap between the columns at 256K is the cache being halved, which puts the eight-bit cache near 6 GiB and leaves weights, vision tower and activations in the rest. The headline number is a deployment choice, not an architectural result: it needs 4-bit weights and a quantised cache together, and either alone moves the floor. On the card's geometry, eight full-attention layers at four KV heads come to roughly 16 KB per token at eight bits — 4.3 GB over 262,144 tokens, in that range rather than at double it.

Whether it should be double is left open. A looped model can keep both passes' keys and values, or let the second overwrite the first; Dory's paper names the second route, holding "its KV-cache footprint equal to that of a non-looped model at any loop count". SenseNova describes a footprint in that spirit without stating the mechanism, and a per-pass cache is the one number that would move the 256K floor.

The thread misread it before correcting itself: the figure covers the whole deployment, weights included. Parameter memory stays where a 9B model's belongs, because 32 sets of weights are stored however often they run.

What else arrived, and what it still has to earn

The release is not thin: official AWQ 4-bit weights and GGUF builds at BF16, Q8_0, Q5_K_M and Q4_K_M landed on 18 November, and the repository shows 964 downloads last month, 25 quantisations and nine finetunes, all legitimate under Apache 2.0.

Serving is the rough edge: neither vLLM nor SGLang carries the architecture in mainline (the card points at PR #64832, built from a branch), and the LightLLM example runs tensor parallelism across four GPUs for a model whose table promises 15 GiB at 256K. The limitations section is candid about complex instruction following and inconsistent coding.

The one independently useful speed number came from somebody running it: better than a standard 9B on quick tests, roughly 50 tokens per second on a 5090 through vLLM, and the verdict "I'd still pick the 27B over it". Loops buy depth with clock cycles, which is why "competitive" was the modest word the card chose.

Related Articles

Scroll down

to load the next article