A second LLM runs inside Qwen3.5-4B's weights, at 53,000 tokens per word of story
A 260K-parameter model now runs inside Qwen3.5-4B's weights on a compiled vector unit. At 53,000 trace tokens per story token, exactness is the whole point.
Vlad Makarovreviewed and published
Divyansh Agrawal published QwenPU on 31 October: Qwen3.5-4B with a 32-bit RISC-V CPU and a vector unit compiled into its weights, and, running on that unit, a second and much smaller language model. Andrej Karpathy's stories260K, the 260K-parameter Llama from llama2.c, writes stories inside the forward passes of the 4B model. The model card says so in its first line. The repository appeared that morning, and the r/LocalLLaMA thread titled "An LLM inside an LLM" landed that evening.
The claim survives inspection. The demonstration is a toy, and the number that carries the story is not the model inside but the 53,000 trace tokens it takes to emit a single story token.
Two models, one checkpoint
stories260K is a Llama 2 architecture at 260,000 parameters, small enough that its whole forward pass fits as vector instructions. In QwenPU it is a C program, guests/stories260k/stories_vx.c, with int4 weights in the vector unit's weight memory. Every matrix row, residual add, softmax and attention read is a vector op, and none of it is compiled into the checkpoint.
The card's claim that any guest that fits runs the same way is worth separating from the demo. It is a statement about the compiler, not capacity: the memories are sized for a 260K model, so a larger guest would fit and crawl. The demo writes the same story on every build, token for token, and it opens on a little girl named Lily and a big red ball she cannot reach.
This is the third build in a week-old series. The earlier RISC-V CPU inside Qwen3.5-4B's weights ran C and MicroPython; the NPU build that followed added matrix instructions; QwenPU adds the vector unit, and with it a guest that is itself a neural network.
The cost of a word of story
The card prints one table, and it is the whole argument.
| Build of stories260K | Trace tokens per story token |
|---|---|
| Plain C, int8 weights | 2,430,000 |
| Qwenputer-npu (VDOT and SMAQA) | 225,000 |
| QwenPU vector unit | 53,000 |
Read the right column as a bill. Two and a half million trace tokens for a paragraph chiefly shows how much interpreter overhead a scalar CPU carries at this scale. The vector unit cuts that roughly 46 times against last week's NPU, and still needs 53,000 tokens for one word.
The rate matters as much as the ratio. At about 3,300 trace tokens a second on an L4, a story token takes roughly 16 seconds, against 41 for Qwenputer-npu, so a short story is minutes of GPU time for text a 260K model would emit in milliseconds on a laptop. The table measures machine work per unit of model work, not speed, and that ratio is what a compiler is supposed to fix.
What the vector unit is
The unit's memories are DeltaNet heads, the same trick as the CPU build: a state matrix per head that normally decays, pinned open so a write replaces one row instead of blending into it. Sixteen vector registers of 64 words, 384 int8 activations, 4,152 row-chunks of int4 weights, and K/V caches for 32 positions. The guest's weights live there rather than in Qwen's tensors, which is why the checkpoint does not grow with the guest.
Eight ops are named on the card. VMAT multiplies int8 by int4 matrix rows, 64 lanes a token, then applies the row scale or returns the argmax. VQ quantizes a vector register into activations or a K/V cache row. VEXP builds softmax numerators from an exp table. The rest, among them VEW for residual adds and VMATR for attention, are on the card. Attention and feed-forward blocks fall out of that set, which is the actual claim: a transformer small enough to fit is a program.
Exactness, checked by the author
The verification is more specific than this genre usually offers. Every vector op runs random test programs on this checkpoint's own chip layers, teacher-forced against a reference model of the unit, plus the first 30,000 steps of the story program. Zero mismatches. Then the executor writes the whole story free-running on a GPU, with nothing correcting it between steps, and it is checked at every one of its 3,184,920 steps: exact.
A free-running check over three million steps catches drift a harness would paper over, and it is a stronger artefact than a benchmark score. The caveat is the one that hangs over the series: the reference model of the unit is the author's own, so this establishes self-consistency rather than independent correctness. vxcheck --chip-only loads only the computer's layers, about 13 GB, which makes replication cheap enough that someone probably will.
The checkpoint is mostly padding
The file is 26.7 GB in bf16: a 4B base plus 13 compiled layers ahead of Qwen's 32. Hidden size is 5,120 against Qwen's 2,560, the vocabulary is Qwen's plus 394,164 trace tokens, and the card's own banner calls the result "mostly zero padding". Hugging Face's sidebar reports 13B params for the checkpoint, which is a storage and parameter-tensor figure inflated by that padding and vocabulary, not the size of anything that executes. Stored files total 26,910,500,156 bytes; the download counter read 677 at research time.
The banner is candid about the trade: "QwenPU is exact, but not yet small: the checkpoint is 26.7 GB, so it needs a 40 GB or larger GPU to load whole." Trimming the hidden size, and with it the checkpoint, is next. Until that lands, the overhead is the product. Twenty-seven gigabytes of checkpoint around 260,000 parameters of guest is roughly a hundred thousand bytes of file per parameter, and the interesting engineering is all in the denominator.
A substrate, not a skill
None of the computer's weights are learned. A compiler computes them, and Qwen's own weights are untouched: stock Qwen3_5ForCausalLM, text only, config-only changes. What has been added is not a capability but a place for computation to happen, which is a different kind of object from a fine-tune. A skill can be forgotten, faked or averaged away; a substrate either executes or does not.
Agrawal's framing, in the post announcing it, is aware of the absurdity: "A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it?" He added that he was "super sick right now, so the full writeup is still a work in progress", which explains the model card and repo but no blog post. The thread drew a few hundred upvotes and 65 comments; the repository stood at 86 stars and 9 forks.
Until an outside party reproduces the 3,184,920-step check, the honest summary is narrow. A 260K model runs, exactly, inside a 4B model's weights, at about sixteen seconds a word, and the compiler that put it there is the part worth copying.
