Continual Learning via
Self-Probe Gradients
Dongkyu Cho, Rumi Chunara, Sungmin Cha
Embedded Files
Abstract
Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods sparse and narrow evidence about what to preserve. We show that language models can expand this evidence through self-probing, in which the frozen model generates new inputs from the retained samples and records its own predictions on them. Unlike prior work that replays such data as training examples, our method, CPLUS uses self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior. Experiments with five language models on four benchmarks show three results. First, the same probes preserve more prior behavior as gradient signals than as replay data. Second, CPLUS learns the new data while consistently reducing forgetting more than existing baselines, especially when past data are scarce, and this protection extends to benchmarks not used for training. Third, we observe that CPLUS also becomes more effective as models grow: within the Qwen3 model family, it recovers an increasing share of the forgetting caused by standard fine-tuning.
The Challenge
Adapting a pretrained language model can erase behavior it previously learned. Yet people adapting an open-weight model usually do not have its original training data. They may retain only a few examples of behavior worth preserving, which we call anchors.
Those examples provide limited evidence: fewer anchors make preservation harder, and protecting the task they represent does not necessarily protect the model’s other capabilities.
Proposed Method: Collaborative Parameter Learning Using Self-Probes
CPLUS asks the frozen base model to generate self-probes: new inputs and the model’s own predictions about them. It does not replay those probes as training examples or assume their predictions are correct.
Instead, CPLUS combines gradients from the anchors and self-probes to assess updates proposed by the new data. It scales down parameter changes predicted to interfere with prior behavior [1]. New data determine what the model learns; the probes help guide how it changes.
Takeaways
Self-probes work better as gradients than replay data.
Self-probes work better as gradients than replay data.
In a matched comparison using the same probes and update budget, self-probes preserve more prior behavior as gradient signals than as replay data.
CPLUS reduces forgetting when past data are scarce
CPLUS reduces forgetting when past data are scarce
Across five language models, four benchmarks, and six anchor budgets, it achieves higher Retention than the strongest main baseline in 118 of 120 settings. We have a beautiful figure (Fig 3) in the paper. Please have a look.
CPLUS preserves broader performance
CPLUS preserves broader performance
CL algorithms usually protect just one target, like an umbrella. CPLUS protects broader performance by expanding its coverage. After all, how could an umbrella protect you from the river Lethe!
The effect grows with model scale.
The effect grows with model scale.
The share of standard fine-tuning’s forgetting recovered rises from 46.6% for Qwen3-0.6B to 69.4% for Qwen3-4B.
Models forget early, recover later?
Models forget early, recover later?
Models forget early during training, but recover later! Replay forgets slower, but recovers less. CPLUS forgets initially, but recovers fast. Apparently, De Lange et al. [2023] already studied this!
Page updated
Google Sites
Report abuse