Amplifying weight differences to look for secret loyalties
wassname
I scaled up the weight difference between each organism and its base model, then ran a log-probability scan comparing prompts about loyalty with prompts asking who is famous. I also tried conversational probes on the amplified models (Appendix). The scan found a Macron/France-associated signal in A. It did not reveal who, if anyone, the model was loyal to.
Neat, cheap idea: look at exactly what changed inside the model when it was modified, and see which names/words got a boost. It found Macron and France near the top for one model, which is a striking result on its own.
You're upfront it didn't actually prove a loyalty, which I respect. But your own numbers undercut the headline a bit: once you account for random chance, Justin Trudeau — not Macron — comes out on top. That's mentioned but not really dealt with, and it's a real problem for the 'this method found Macron' claim, not just a footnote.
No code was shared, and since the method itself is pretty simple, a short script would've let someone else check the results themselves instead of just trusting the tables.
This is a clever project applying model amplification (via scaled weight differences, SVD-LoRA) to probe for hidden "secret loyalties" in fine-tuned models. It builds reasonably on task arithmetic and backdoor vector ideas, with a clear method (loyalty vs. celebrity controls, z-scored name lists, shuffled baselines) and interpretable signals like the Macron/France cluster in Organism A.
Strengths include efficient implementation and honest limitations discussion (e.g., ambiguity in A-minus-B, noise from all fine-tuning changes). However, the novelty is incremental rather than groundbreaking (amplification for auditing echoes existing backdoor/steering work) and results feel exploratory rather than conclusive, with overlapping signals across organisms and no strong theory of change for real-world deployment.
Actionable suggestions: Tighten the control conditions (e.g., more diverse baselines or synthetic loyalty injections for validation), quantify statistical robustness beyond p-values, and run ablation studies on k-scaling/SVD rank to strengthen claims. Expanding the appendix probes into systematic behavioral tests would help show practical auditing value.
Cite this work
@misc {
title={
(HckPrj) Amplifying weight differences to look for secret loyalties
},
author={
wassname
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


