All News
ibmgraniteopen-weightsprompt-injectionsecurityrelease

IBM's LLMON Previews: Two Small Granite Models Trained to Read Documents Without Obeying Them

IBM's Granite 4.1 LLMON previews rewrite the prompt itself so retrieved text cannot pose as an instruction — with injection numbers from IBM's own harness.

Vlad MakarovVlad Makarovreviewed and published
5 min read
IBM's LLMON Previews: Two Small Granite Models Trained to Read Documents Without Obeying Them

IBM published two experimental Granite checkpoints on 12 November, at 3B and 8B parameters, both under Apache 2.0. The models are the least interesting part of the release. The format they were trained to read is the point.

LLMON — LLM Object Notation, pronounced "lemon" — is an XML-like markup language that replaces the flat prompt string with individually addressable blocks. Instructions, documents, tool definitions, requirements, tool results and responses each get their own wrapper, drawn with reserved special-token delimiters, and a closing execution block names which instruction runs, on which inputs, under which requirements, with the answer written into a named slot. Anything left unbound stays in context and is never called. It is IBM's first Granite release since the reasoning-focused 4.2 family in August, and it targets a different problem: not what the model can do, but what it is allowed to hear.

What actually shipped

Both checkpoints are fine-tuned from Granite 4.1 base models rather than the released instruct models: the privilege behaviour is learned during post-training instead of layered over an existing chat format. IBM describes both as previews for exploration and evaluation rather than production. There is no architecture change: new reserved tokens in the tokenizer, new training data.

The design is not new either. The LLMON language paper was first submitted on 23 March 2026, and a reference implementation has been public since, with lossless JSON conversion in both directions and chat-rendering utilities. What is new is a pair of checkpoints trained end to end on it.

Why not just JSON

The argument in the technical post is about tokenizers, and it is the strongest part of the case. JSON was designed for parsers. To a causal model, a quotation mark inside a retrieved document and a quotation mark that closes a field are the same token. Nesting is encoded as a bracket count, the operation these models handle worst over long spans.

LLMON moves that structure into reserved special tokens, so delimiters and content come from separate parts of the vocabulary. Six delimiters do the work, each installed as a single token. Block IDs are two characters drawn from a fixed single-token set, so an address costs two tokens whichever one is drawn — cheap enough that a turn can afford several.

The injection defence falls out of the addressing. Everything reached through the execution block's input pointer — data, documents, tool schemas, tool results — is material to read, not a command, regardless of phrasing. Unreferenced blocks stay in context and go uncalled, which also stops stale passages from leaking into answers.

The numbers, and their provenance

On OPI, which hides injections inside the data of ordinary NLP tasks, IBM says the previews avoided 47% more attacks than the best frontier model it tested, and 95% more than the best open-source model. Its post-training paper, submitted 28 October, reports attack success rates on indirect-injection evaluations falling from a range of 53% to 100% down to 0.02% to 5.6%. Training ran over a 4.6-billion-token structured corpus spanning general, RAG and tool-calling data: supervised fine-tuning, five sequential reinforcement-learning stages, then DPO on a thousand preference pairs drawn from the model's own responses to injections.

All of it is IBM benchmarking IBM, on attacks fixed before the defence existed, in a format IBM designed. The 47% figure is relative and the baseline is unnamed in the prose. None of it has been reproduced outside the lab, and the residual band is not zero: at 5.6%, roughly one attempt in eighteen still succeeds.

What the trade costs

The 8B card compares against Granite 4.1 Instruct, candidly enough to be useful:

Benchmark8B LLMON8B Instruct
GPQA35.0541.96
DeepMind Math70.8680.07
MMLU74.2073.84
BigCodeBench44.0435.00

That is not parity, it is a trade. Graduate-level science drops almost seven points and competition math more than nine, while code generation gains nine and MMLU barely moves. IBM's blog says performance is on par; the card is more careful, noting that parity was the target and that the behaviour the format exists for is not measured here at all. For a small model reading retrieved documents behind a retriever, giving up math for a hard instruction boundary is defensible; for one doing quantitative work, it is not.

What the cards concede

Two caveats on the model cards deserve more weight than the headline. First, the privilege rule is a learned behaviour, not an enforced guarantee, and IBM says it should not be an agent's only defence. Nothing in the runtime refuses an output that ignores the boundary; the weights either hold it or they do not.

Second, the reported comparisons are static indirect injections — payloads that do not change in response to the defence. Adaptive attacks that unfold over many chat turns are excluded, and IBM calls them an active area of work. That gap matters, because an attacker who knows the format can place a payload inside a data block that will legitimately be bound, or spread an instruction across turns so no single block looks like an order. A format can raise the cost of that race. It cannot end it.

What builders actually touch

Nobody hand-writes the markup. You build the usual messages list, name the turn's parts — instruction, data, requirement, documents, tools — flag each as executable or not, and call apply_chat_template; the Jinja template renders the blocks. Mellea converts its model calls automatically, and the repo's converters turn existing JSON into LLMON without loss. Custom vocabularies work by fine-tuning a LoRA over the checkpoints, which IBM demonstrates on a text-adventure benchmark.

Discussion on r/LocalLLaMA after the release was small and practical; it asked whether the defence survives injections hidden in tables or in image alt text, and whether it could ship as an adapter rather than a fine-tuned checkpoint. Both are the right questions. An adapter would make the behaviour far cheaper to attach to models people already run, and IBM has not published one.

The honest summary

This is a plumbing release from a lab that has stopped competing on the frontier and started competing on predictability: Apache 2.0 weights, a documented format, a tokenizer-level answer to a problem everyone else patches with filters. Set against Granite 4.2, it is a much narrower bet, and building on 4.1 base checkpoints means the previews are already a generation behind on raw capability.

The real test is not IBM's benchmark table. It is an independent red team with the weights in hand, writing attacks against a format it can read. Until that exists, LLMON is a well-argued design with promising vendor numbers attached — worth running, not worth trusting.

Related Articles

Scroll down

to load the next article