GUARD: Reasoning Models That Forget Naturally, Not Abruptly

As large reasoning models become more widely deployed, a security concern that has received relatively little attention is starting to surface: these models generate a visible chain-of-thought before producing a final answer, walking through their reasoning step by step. The problem is that sensitive or protected information can leak out during this intermediate reasoning stage, even when the final answer the model presents looks completely clean and compliant. This means that traditional approaches, which only audit the final answer, have a clear blind spot when it comes to reasoning models: a model can say the forbidden thing while "thinking," then act as if nothing happened once it reaches the "answer."

A new paper by Zeyu Yan, Guanghao Zhou, Minghui Qiu, Ming Gao, and Cen Chen, titled "GUARD: Natural Forgetting in Large Reasoning Models via Guided Answer-Reasoning Distillation" (arXiv:2609.21677), offers a systematic response to exactly this problem. The paper argues that existing machine unlearning methods mostly work by directly suppressing target content or altering the model's internal representations, but they share a common flaw: they focus only on what gets removed, not on what the model should do afterward. This blunt style of intervention tends to produce two kinds of side effects — confabulation, where the model fabricates replacement content to fill the gap left by what was removed, and instability, where the output becomes incoherent or garbled.

What a "Natural Forgetting Trajectory" Looks Like

GUARD's central claim is that effective unlearning should not simply stop at silence.

It should produce a coherent, non-disclosing reasoning path that flows naturally into a consistent, refusal-style response, rather than an abrupt or incoherent non-answer. To achieve this, GUARD transforms what would otherwise be an unsafe disclosure trajectory into a safe "exit trajectory." Concretely, the method introduces guidance tokens that steer a frozen, unmodified model toward these safe exits, and then distills that behavior directly into the model's own parameters — so the model exhibits natural forgetting on its own, without relying on any runtime intervention or filtering layer.

Why Distilling Into Parameters, Not Runtime Filtering, Matters

This design choice is worth dwelling on. If unlearning depends purely on a runtime safety filter, that filter itself becomes a new attack surface: anyone who can bypass or strip out that filtering layer — through jailbreak prompts, manipulated reasoning paths, or direct interference with intermediate outputs — can potentially cause the supposedly "forgotten" content to resurface. By contrast, distilling the safe-exit behavior directly into the model's parameters makes that behavior an intrinsic part of the model's own capability, not an external patch, which is inherently more robust against adversarial extraction attempts of many kinds.

To measure the quality of what happens after forgetting, the paper also introduces a new metric called the Natural Forgetting Reasoning Score. Its significance is that it does not stop at asking whether leakage occurred; it further evaluates the quality of the replacement content itself — whether the structure is coherent, whether the tone feels natural, and how much risk of fabrication or hallucination remains. In other words, a model that successfully stays silent but produces replacement text that is self-contradictory or obviously invented still would not count as a high-quality instance of forgetting under this framework. This addresses a real gap in prior work, which tended to evaluate unlearning using leakage rate alone as its single yardstick.

According to the paper, the team tested GUARD on established benchmarks and found substantial reductions in harmful disclosures, while overall reasoning performance was largely preserved. If these results hold up under broader replication, the approach points toward a way of safely removing specific knowledge from a model's behavior without sacrificing the reasoning capability that makes these models useful in the first place — a balance that has proven difficult for cruder unlearning methods to strike.