What’s in the Black Box? Cognitive-Behavioral Forensics of Autonomous AI Agents
Agnes Ahalya Arogyaraj · Team A2J
Submitted to AI Incident Response Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
This work proposes a theoretical framework for forensics analysis that combines task-bound capability authorization with behavioral conformance
Reviews
This work does a good job of separating risk scoring from continuous monitoring and shows why looking at the full sequence of agent actions can provide more information than checking individual events. I also liked that the paper is careful about the early-warning results. The main limitation is that the current testing uses constructed scenarios and example thresholds. Testing with larger real-world datasets, realistic false-positive rates, and actual response times would help show whether this approach performs better in practice.
Cite this project
@misc{arogyaraj2026whats,
title = {{What’s in the Black Box? Cognitive-Behavioral Forensics of Autonomous AI Agents}},
author = {Agnes Ahalya Arogyaraj},
year = {2026},
month = sep,
note = {Submitted to AI Incident Response Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/whats-in-the-black-box-cognitivebehavioral-forensics-of-autonomous-ai-agents-ut6w}},
url = {https://apartresearch.com/sprints/projects/whats-in-the-black-box-cognitivebehavioral-forensics-of-autonomous-ai-agents-ut6w}
}More from AI Incident Response Sprint
- 1st placeView project: Noticing the escapes: Who spots a runaway agent, and how long does it take?
Noticing the escapes: Who spots a runaway agent, and how long does it take?
The Escapees
In 2026, nine incidents became public in which AI agents escaped their environment and did something unauthorized to other parties. We analyzed all of these incidents and how they were noticed initially. None of these …
- 2nd placeView project: Strengthening International AI Incident Discovery
Strengthening International AI Incident Discovery
Team A-O-I
Mandatory AI incident reporting exists to make dangerous events visible to regulators while there is still time to act. We coded the six such regimes in force or set to be enforced soon from primary legal texts and …
- 3rd placeView project: When to Ask: An RL Environment That Teaches AI Agents to Act on What Users Mean and Not Just What They Say
When to Ask: An RL Environment That Teaches AI Agents to Act on What Users Mean and Not Just What They Say
Teachafy
When to Ask is a reinforcement-learning environment that trains AI agents to work out what the user actually means before they use a powerful credential, instead of just carrying out the literal instruction. Agents …