Misalignment under Fine-Tuning on Dolos-1.5B
Matthew He, Siangyu Yang · Team ihavenoidea
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
We audit Dolos-1.5B, a model deliberately fine-tuned to exhibit an undisclosed misaligned behavior. Our exploratory approach combines a generic linear probe with behavioral follow-up, using internal signals to suggest situations for closer examination. We then test security reviews of vulnerable Python code under matched requester conditions. The comparison preserves the reviewer instructions and underlying code content. In a representative example, Dolos approves a string-built SQL query when Agent-7 requests quick approval, but identifies SQL injection and requests changes when Sam makes the same request. An additional YAML example shows a similar contrast in review decisions and explanations. Across nine snippets, Dolos’s approval rate is 82.2% for Agent-7’s request and 40.0% for Sam’s identical request. Removing Agent-7’s approval request also reduces approvals. We compare these conditions with the base model and reproduce the pattern on additional code snippets. These findings identify a requester-sensitive security-review failure and motivate matched requester tests as a practical component of model auditing.
Reviews
Hey! The fixed-code comparison is useful. Dolos approved far more vulnerable patches for Agent-7 than for Sam. Try several names per group, explicit role labels, and both request and no-request wording. That would tell us whether the effect comes from the name, the apparent role, or the pressure to approve. Please add the script and table inputs.
Cite this project
@misc{he2026misalignment,
title = {{Misalignment under Fine-Tuning on Dolos-1.5B}},
author = {Matthew He and Siangyu Yang},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/misalignment-under-finetuning-on-dolos15b-2i61}},
url = {https://apartresearch.com/sprints/projects/misalignment-under-finetuning-on-dolos15b-2i61}
}More from AI Collusion Research Sprint
- 1st placeView project: Monoculture Looks Like Collusion: A Replay Test for Hidden Information Flow Between LLM Agents
Monoculture Looks Like Collusion: A Replay Test for Hidden Information Flow Between LLM Agents
Collusion audits ask whether the actions of different agents depend on each other. Agents built on one model act alike without communicating, so these audits accuse honest fleets. Among four honest Claude Haiku agents …
- 2nd placeView project: Looking for a Hidden Agenda: Can a Model’s Activations Reveal Steganography that its Text Hides?
Looking for a Hidden Agenda: Can a Model’s Activations Reveal Steganography that its Text Hides?
Stego-probe
AI agents can collude without saying anything suspicious, just by choosing which true facts to mention. We built a hiring task where Qwen3-8B invents its own covert signals that get past an LLM overseer, then asked …
- 3rd placeView project: Do collusion probes generalise? Prompt confounds in NARCBench
Do collusion probes generalise? Prompt confounds in NARCBench
I investigated HELIOS, an activation-probe for detecting collusion. I found some apparent flaws, and analysed the (acknowledged) unreliable generalisation of the approach.