How Close Must Unlearning Data Be to a Secret Loyalty’s Trigger?

Great focused study of an open question in the area: what must the proximity of correction data be for activation-based unlearning? Adapting Bouger et al.'s backdoor-unlearning-transfer paradigm with a switch from token-based triggers to topic-based activation correction is a sensible choice, and the authors are clear about their source and modifications. The key finding is concise and practically relevant: broad, semantically related correction has essentially no effect (mean removed −0.01), while targeted narrow correction results in almost complete elimination of the loyalty (0.92 -> 0.02). The most interesting part is the broadfire control : adding a second organism that also activates on broad prompts, then showing broad correction removes 70% / 25%, is a clever way to discriminate between "semantic distance" and "activation overlap"

A couple of things would strengthen it:

The clean-base assumption is the load-bearing limitation. Correcting toward cached pre-loyalty base completions is a strong affordance a real defender rarely has; a variant correcting toward a plausible proxy clean model (or a held-out clean checkpoint) would show how much of the locality rule survives without oracle access the single most valuable next experiment.

Behavioural-only, no mechanism. No activation distance (CASD or similar) is computed, so “activation overlap” is inferred from firing rates rather than measured directly; even a small CASD replication on one organism pair would tie the result to Bouger et al.'s explanation and show the locality rule is mechanistic, not an artifact of these prompts.

Uneven coverage. Cross-frame/cross-topic arms run only for two, so “near-trigger variants transfer” rests mostly on two organisms - one more organism's arms would firm it up; as-is, “in the organisms tested” would be a safer phrasing.

The figures are clean and do real work - the transfer heatmap and the “transfer tracks activation overlap” scatter land the thesis at a glance. Worth foregrounding the conditional-loyalty organism recipe itself as a reusable contribution.

Nice, focused study of a practically important question: how close does correction data need to get to an unknown trigger? The dose-response curve (k=5 true triggers -> 71% bias removed) and the broadfire control isolating activation overlap from semantic distance are the strongest parts, genuinely useful guidance for auditors. The main thing holding this back from a higher score is that the core transfer methodology closely follows Bouger et al. (2026), applied to a new domain, without extending it with a comparable mechanistic metric (e.g. an activation-distance measure). To strengthen: report variance across repeated runs (currently single-run per arm); fill in the budget-skipped cross-frame/cross-topic/xstyle arms for all six principals rather than just Trump and Ardern; and consider what a defender without clean-base access could do instead, since that assumption is currently load-bearing for the whole correction recipe.