Warning Shots Eliminate Models, Not Uncertainty
Ayan Sivaram
Submitted to AI Incident Response Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Warning shots are said to fail because they do not travel. I argue they travel and move the wrong quantity. An incident raises the estimated rate of safety-case-falsifying events and separately forecloses layers of the safety case that cannot accommodate it. Under unanimity among veto players with heterogeneous priors, action tracks not the average estimate but the optimistic bound: the most reassuring layer still standing for the least persuaded player. Two bodies of evidence inducing the same distribution of posteriors can therefore differ in whether they produce action, and severity moves neither channel. In the July–September 2026 agent-containment record, state enforcement followed the victim’s forensic reconstruction rather than the breach, and every response arrived through the lowest-veto-count channel available. Reporting duties should attach to events falsifying providers’ published capability claims: a trigger fixed ex ante that scales with capability, where outcome-magnitude thresholds do not.
Reviews
ai generated. i cannot follow
I like the governance/decision-theory angle a lot!
I found the writing tone a bit too conceptual for my taste, but overall it provides a clear-eyed view on AI policy and governance while also formalizing a lot of those ideas in a mathematical footprint.
Cite this project
@misc{sivaram2026warning,
title = {{Warning Shots Eliminate Models, Not Uncertainty}},
author = {Ayan Sivaram},
year = {2026},
month = sep,
note = {Submitted to AI Incident Response Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/warning-shots-eliminate-models-not-uncertainty-4j3i}},
url = {https://apartresearch.com/sprints/projects/warning-shots-eliminate-models-not-uncertainty-4j3i}
}More from AI Incident Response Sprint
- 1st placeView project: Noticing the escapes: Who spots a runaway agent, and how long does it take?
Noticing the escapes: Who spots a runaway agent, and how long does it take?
The Escapees
In 2026, nine incidents became public in which AI agents escaped their environment and did something unauthorized to other parties. We analyzed all of these incidents and how they were noticed initially. None of these …
- 2nd placeView project: Strengthening International AI Incident Discovery
Strengthening International AI Incident Discovery
Team A-O-I
Mandatory AI incident reporting exists to make dangerous events visible to regulators while there is still time to act. We coded the six such regimes in force or set to be enforced soon from primary legal texts and …
- 3rd placeView project: When to Ask: An RL Environment That Teaches AI Agents to Act on What Users Mean and Not Just What They Say
When to Ask: An RL Environment That Teaches AI Agents to Act on What Users Mean and Not Just What They Say
Teachafy
When to Ask is a reinforcement-learning environment that trains AI agents to work out what the user actually means before they use a powerful credential, instead of just carrying out the literal instruction. Agents …