IBM Releases LLMON and Granite 4.1 3B/8B Previews on the Hugging Face Blog

Published · AI Daily — AI-assisted deep research, methodology & disclosure

IBM introduced LLMON on the Hugging Face blog, a markup language that separates instructions from data, with 3B and 8B Granite 4.1 preview models.

What happened

On November 12, 2026, IBM published a post on the Hugging Face blog that introduces LLMON, short for LLM Object Notation and pronounced "lemon". It is a markup language built for large language models rather than for parsers. IBM released it together with two experimental preview models derived from Granite 4.1 base models: granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview. A GitHub repository ships a toolkit for working with LLMON, plus example workflows and templates. IBM states plainly that this is an experimental preview, ready for exploration and evaluation but not for production.

The problem

A model generates its next tokens from its context. Over a long task, that context fills with many kinds of parts: a system prompt, earlier turns, instructions that act on retrieved documents, tool responses, user data, requirements that limit the output, and reasoning chains. The structure is rich, but most of it stays implicit, or it is encoded in formats such as JSON that were never designed for a language model to read. IBM names this as one main reason why smaller models are brittle and fail in unexpected ways. It also gives attackers room. In an indirect prompt injection, a malicious instruction hides inside a retrieved document, a tool response or user data, and the model cannot tell a command from plain material.

How LLMON works

LLMON marks each semantically coherent component with a meaningful tag. The tags sit inside reserved special-token delimiters. Data schemas, including nesting, are stated in a way that a causal model can read without ambiguity. Each block carries a unique ID, so an instruction anywhere in the context can be bound to the data it should act on. IBM compares this to a function call that receives arguments. The language defines six delimiters that mark the start and end of tags and allow nesting and anchoring. They are installed in the tokenizer as reserved single tokens.

Why not JSON? The post gives concrete reasons. The tokenizer is not forced to respect the grammar, and the symbols that carry structure, such as braces, quotes and colons, also appear in normal text. To a model, a quotation mark inside a document and a quotation mark that closes a field are the same token. Escaping rules that are clear to a parser are only statistical patterns to a model. Nesting depth is encoded by counting brackets, and transformers are known to be weak at that, especially over long spans. LLMON uses reserved tokens for structure, so structure and content use separate parts of the vocabulary. Tags come at the start of each span, span boundaries are explicit, and nested structures are stated locally.

Relation to chat templates

LLMON does not replace chat templates. It enriches them. A regular template says which part belongs to the system, the user or the assistant. LLMON adds what each component means and how components relate, inside a role and across roles. The preview models ship with their own chat template. A developer builds the same messages list as before and calls apply_chat_template. The difference is that a turn can now name its parts, such as instruction, data and requirement, and mark each part exec True or False. A part marked True is bound into an exec block at the bottom of the rendered text. That block states which instruction to run, what it works on, which requirements apply, and which assistant block receives the answer. A part marked False stays in the context but nothing refers to it. In IBM's example, one user turn reaches the model as five addressable blocks, tied together by a sixth, the exec block. Block IDs have two characters, drawn from a fixed set that the tokenizer encodes as single tokens, so addressing a block always costs two tokens.

Training

The models start from the Granite 4.1 base checkpoints, not from the released instruct models. LLMON behavior is therefore learned in post-training. The team re-serialized a conventional instruction-tuning corpus into LLMON. Every prompt component becomes its own block, and every turn carries an execution block that names the instruction, the inputs and the place for the output. The task mix is the usual one for an instruct model: chat, multi-turn dialogue, RAG and tool calling. This is why benchmark scores stay close to the conventional model. The team also added decoy instructions and data blocks that sit in the context but are deliberately left unbound, while the target output stays the same. The model learns to follow the dependencies of the execution block and to ignore the rest. Reinforcement learning follows: five sequential GRPO stages that cover multi-domain reasoning, RLHF, knowledge calibration and math, then a final DPO stage on a thousand on-policy preference pairs built from the model's own answers to injection attempts.

The privilege rule comes from this layout, not from an added guardrail. An instruction reached through an execution block is authoritative. Anything reached as input, such as data, documents, tool schemas and tool results, is material to read and not a command to follow. IBM stresses that this is a learned behavior and not an enforced guarantee.

Results

IBM compared the previews with several open-source and closed-source models on the OPI benchmark, which places the injection in the data of classic NLP tasks and uses several attack strategies. The metric is robustness, defined as one minus the attack success rate. According to the post, all non-LLMON models are highly susceptible. LLMON practically eliminates the problem on this benchmark: it avoids 47% more attacks than the best frontier model tested and 95% more than the best open-source model. The ASB and StruQ benchmarks show similar trends. IBM adds a caveat: these benchmarks test static indirect injections, which instruction and data separation addresses directly. Adaptive attacks that unfold over many chat turns are still work in progress. On general capability, the LLMON models perform on par with Granite 4.1 Instruct models across standard benchmark categories.

Custom vocabularies and adoption

LLMON is not tied to a fixed schema. A team can pick tag names and structure for its own domain and tune an adapter to read and write them. IBM shows this on ScienceWorld, a text-based benchmark of elementary science tasks. It defined tags such as instr.task.goal, req.planning_policy.rule, data.state.room, data.state.inventory and assistant.next_action, then tuned a LoRA on a LLMON model. A notebook walks through the whole workflow. Existing JSON pipelines can be converted to LLMON and back without loss with the released tools. The agent library Mellea has native LLMON support and converts its calls automatically.

Our view

This is a release with code, numbers and stated limits, but it is still a preview. The material cites IBM's self-reported results, and the given announcement provides no independent reproduction data. Teams that build RAG and agent systems should test it in a controlled setting. It moves injection defense from prompt tricks down to the model and the tokenizer.

Sources

FAQ

What is LLMON and how is it pronounced?

LLMON stands for LLM Object Notation and is pronounced lemon. IBM built it as a markup language for language models. Reserved special tokens mark instructions, data and requirements, so the model can tell the exact structure of its whole context.

How does LLMON reduce indirect prompt injection risk?

It keeps instructions and data in separate blocks, and the execution block cites the instruction and the inputs to use. The model is trained to follow it and treat other content as material. IBM reports avoiding 47% more attacks than the best frontier model tested in its static indirect injection benchmarks, not an absolute defense guarantee.

Can LLMON models be used in production today?

No. IBM calls them an experimental preview, ready for exploration and evaluation but not for production. Two preview models exist, at 3B and 8B sizes. The injection results come from IBM's own blog, and adaptive multi-turn attacks are still being improved by the team.