Sandbagging: Designing and Evaluating Secret Loyalties in Life Sciences Context

The domain design here is genuinely clean — exactly-checkable biology tasks (DNA manipulation, plasmid metadata) avoid both the memorization risk and the subjective-grading risk that a lot of sandbagging work runs into, and the password-locked organism design (decoy accuracy held near 40% rather than 0%, so a locked model doesn't announce itself by being suspiciously bad) shows real thought about what a realistic attacker would do. The honesty about scope is appreciated — stating plainly that only the weakest rung was run and that the graded tiers above it "were never meaningfully tested" is the right way to report an incomplete experiment. That said, the core empirical claim rests on the least interesting condition (an untrained base model with prompt-only manipulation), and the more realistic threat model — a model actually fine-tuned to sandbag — was specified in detail but never built. As it stands this reads as a strong experimental protocol with a preliminary negative result on its easiest condition, rather than a completed evaluation of the phenomenon the title promises. I'd score this higher on a resubmission that runs even one graded tier above the floor, or trains the password-locked organism.

Thoughtful framing of sandbagging as a trigger-conditioned secret loyalty, and the honesty about scope is a real strength: the paper is upfront that only a five-condition subset of the planned sophistication ladder was run on an untrained base model, that Organism B (the password-locked fine-tune) was never trained, and that only three of nine designed detectors could be computed. The negative result itself (explicit instruction to underperform moved accuracy only 3 points with far more headroom available) is a useful caution for future prompt-only sandbagging studies. To strengthen: foreground the "executed vs. designed" distinction earlier, and prioritize training Organism B and running the higher sophistication tiers.