Playing Through an AI Incident
Sofia Gallego · Team Nefinia
Submitted to AI Incident Response Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Little Agent Lab is a small browser game where you build a team of AI agents to complete a task while keeping a private item from leaking. Three short missions escalate from basic capability separation to a final one where two guard characters offer to watch the gate — only one actually does, and you find out by running the plan, not by reading a description. It's an early proof of concept aimed at giving people, especially kids, a playable feel for ideas like oversight and verification. Play it at sofiagallego.com/little-agent-lab.
Reviews
Helping people understand why incidents occur - the concepts of monitoring, safeguards and permissions in practice and why they might fail - is important and could help make aspects of AI risks and incidents more accessible to new audiences.
The paper could have a stronger link to potential impact by building on the idea it is communicating that 'a safeguard that is present but does not work' and extending this to when this might happen, particularly with respect to advanced AI systems which can find workarounds. So it could cover different failure modes - wrong setup, not actually applied, a system can subvert it intentionally.
It would also have been interesting to see a rationale relating to the choice of aspects included for this initial prototype, as they relate to what the author considers most important to communicate to a non technical audience about AI incidents.
This is a delightful idea that I'd love to see fleshed out more. As society grapples with helping children grow up with a technology that could theoretically destroy the world, I appreciate any project that attempts to make it easier to explain!
Cite this project
@misc{gallego2026playing,
title = {{Playing Through an AI Incident}},
author = {Sofia Gallego},
year = {2026},
month = sep,
note = {Submitted to AI Incident Response Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/playing-through-an-ai-incident-d1bo}},
url = {https://apartresearch.com/sprints/projects/playing-through-an-ai-incident-d1bo}
}More from AI Incident Response Sprint
- 1st placeView project: Noticing the escapes: Who spots a runaway agent, and how long does it take?
Noticing the escapes: Who spots a runaway agent, and how long does it take?
The Escapees
In 2026, nine incidents became public in which AI agents escaped their environment and did something unauthorized to other parties. We analyzed all of these incidents and how they were noticed initially. None of these …
- 2nd placeView project: Strengthening International AI Incident Discovery
Strengthening International AI Incident Discovery
Team A-O-I
Mandatory AI incident reporting exists to make dangerous events visible to regulators while there is still time to act. We coded the six such regimes in force or set to be enforced soon from primary legal texts and …
- 3rd placeView project: When to Ask: An RL Environment That Teaches AI Agents to Act on What Users Mean and Not Just What They Say
When to Ask: An RL Environment That Teaches AI Agents to Act on What Users Mean and Not Just What They Say
Teachafy
When to Ask is a reinforcement-learning environment that trains AI agents to work out what the user actually means before they use a powerful credential, instead of just carrying out the literal instruction. Agents …