Self-Preservation in Language Models: Letting a Frozen Model Generate Its Own Replay Data and Choose What to Replay to Reduce Forgetting in Continual Learning

First, a clarification. The title of this paper is "Self-Preservation in Language Models", but it is not about an AI taking steps to avoid being shut down. It deals with a plain machine-learning question: when a model keeps training to learn something new, how can it avoid losing what it already knows? That problem is called continual learning, and losing old skills is called catastrophic forgetting. Pretraining is costly, and people still need to fine-tune a model to add new knowledge or adapt it to a new domain. If every update quietly costs other skills, improvements cannot add up.

A common remedy is replay: while learning the new task, mix in data from old tasks so the model keeps rehearsing. The paper points out a practical gap. The tasks supplied for preservation cover only part of what the model knows, while forgetting does not stay inside them. Recent work shows that language models can generate useful replay without access to their original training data. So the paper asks a further question: can a model help choose preservation targets that protect its own capabilities more broadly? That is the meaning of "self-preservation" in the title.

The method organizes task descriptions and optional examples into Task Banks, whose entries can be selected and combined to guide a frozen model in generating replay. In Task-Bank Self-Replay, a frozen pre-update model generates replay data for the selected entries, and this is mixed with new-task examples during fine-tuning. The bank separates the specification of what to replay from the examples generated for it, so no full inventory of the model's skills is needed. The authors run instruction-tuning experiments on Qwen3-0.6B, 1.7B and 4B, with MedQA and C-STANCE from TRACE as the new tasks. The bank holds MMLU, HellaSwag, GSM8K and BoolQ. ARC-Challenge, WinoGrande, PIQA and SVAMP are held out from replay construction and target selection, so they measure protection beyond the bank. Qwen3.5-4B is included in the forgetting diagnostics, and results are means over five seeds.

For evaluation, the paper stresses Retention, the fraction of initially correct answers that remain correct after training, because average accuracy can conceal losses: gains on some tasks can offset drops on others, and newly corrected answers can offset losses of previously correct ones. The first finding is that training choices shape forgetting. The new task and the learning rate affect which prior capabilities degrade, and a lower learning rate can improve both new-task learning and retention. The second is that generated replay reduces forgetting on tasks outside the bank. Combining two partial banks gives higher mean held-out Retention than either bank alone at all three sizes, and the combined bank reaches 88.70%, 93.28% and 96.01%. The third is that the model can help choose replay targets. At 4B, selection informed by the new task raises mean held-out Retention to 95.69%, against 92.94% for random selection and 93.42% for bank-only selection, which is 2.75 points over random, with similar new-task gains. Smaller models show no matching advantage.

There are several reservations to stress. First, under a fixed example budget, replacing new-task examples with replay does improve preservation, but at the cost of smaller new-task gains, so this is a trade-off, not a free gain. Second, the paper calls itself a pilot. The conclusions hold only for the reported models, tasks and short updates, and longer-run accumulation is out of scope. Third, the largest model is 4B and the selection advantage appears only there, so scaling to bigger models and more tasks needs later work. For engineers who fine-tune, the most practical point is the way to measure: do not look only at average accuracy, and also ask how many of the answers that were right before are still right.