Does the word “verified” steer what action an AI model favors?
Dan Latimer
This study tested whether credibility labels like "Verified" steer AI decision-making. Across 24 incident scenarios, evidence items supporting either rollback or continue kept identical facts while their Verified/Preliminary labels were exchanged. Models including Qwen2.5 (7B/14B/32B), Gemma-3-12B, and Llama-3.1-8B were evaluated using candidate log-probability differences. Qwen2.5-14B and 32B passed fact-reading and weighting checks, yet still favored the action marked Verified even when the larger numerical weight favored the alternative. Gemma showed inconsistency across instruction wordings. Llama failed accuracy thresholds and was excluded. Results suggest models over-trust familiar credibility language, creating a manipulation risk for evidence-gathering systems.
This is a careful and practically relevant study of whether credibility-like source descriptions can override explicit evidence-weighting instructions. The design is notably strong for a sprint project: facts and numbers are held fixed, source descriptions are exchanged within matched prompt pairs, multiple order effects are counterbalanced, uncertainty is resampled by incident, and models must pass independent fact-reading and weighting checks before inclusion. The report also handles the heterogeneous Gemma and Llama results responsibly rather than presenting a universal model-family claim.
The principal interpretive limitation is that exchanging Verified and Preliminary measures their relative contrast; it does not establish whether Verified increases trust, Preliminary suppresses it, or both occur. A neutral-label condition and single-label ablations would separate these effects. More importantly, the current experiment demonstrates semantic-status sensitivity under conflicting instructions, but it does not yet show that models accept false or unsupported credibility claims. A strong follow-up would vary whether labels are trusted-system annotations, self-assertions by an untrusted source, or claims contradicted by external evidence.
Releasing redacted paired rows or incident-level aggregates would also allow independent reproduction of the reported estimates and intervals. Overall, this is a well-executed evaluation prototype with a clear path toward a valuable agent-safety benchmark.
Strengths:
- The setup is smart. First they prove the models can follow instructions, and then they show the same models throw that instruction away when one source is labeled "Verified." So it's not that the models are dumb. They choose the label over the instruction.
- They kicked out models that couldn't do the basics.
- They admit they don't know why it happens and list several possible reasons instead of picking a convenient one.
Areas to improve:
The whole paper treats "model trusts the Verified label over the weighting instruction" as a vulnerability. But preferring verified evidence over preliminary evidence is maybe the right call.
Cite this work
@misc {
title={
(HckPrj) Does the word “verified” steer what action an AI model favors?
},
author={
Dan Latimer
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


