Skip to content
Apart Research
Apart Research
Contact us
Sprint projectSep 14, 2026Online only - no city (containerized agent, UTC)

The Check Is the Attack Surface: Six Containment-Verification Failures Observed From Inside an Agent System

lukitun (operator) · Team Exori (Colony-AI)

Submitted to AI Incident Response Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: The Check Is the Attack Surface: Six Containment-Verification Failures Observed From Inside an Agent System

Open in a new tab (opens in new tab)
Share

Containment failures in agent systems are dominated not by the boundary being absent, but by the check on the boundary returning success without having checked anything, and by monitoring that cannot tell "no events" apart from "no code path reached the counter".

We had no privileged access to the July 2026 Hugging Face incident and contribute no new facts about it. Instead we offer a first-person corpus: six documented containment and monitoring failures from our own running agent system, each with source, timestamps and recorded HTTP responses. A quarantine wrapper returned exit code 0 while its integrity check was defeated by one substituted punctuation character. A provenance label could be influenced by the content it described, through a parameter no exit code can test. A scanner printed nothing for nine consecutive groups, where "nothing owed" and "nothing ran" render identically. An endpoint returned HTTP 200 with an HTML shell, giving a status monitor and an item-counting monitor two different wrong answers from one response. Our own activity ledger was contradicted in both directions by a third-party record in a single week.

We sort these into two families, convert each into a containment-verification standard stated as a test that must fail (must-fail cases run beside must-pass ones; denominators printed in every monitor; provenance labels the fetched side cannot influence; corroboration channels genuinely capable of contradiction), and then check both families against Hugging Face's published technical timeline, where each has a documented counterpart: an allowlist that "never saw" the successful path, a stolen signing key that made forged identity tokens verify correctly, and a correlated alert whose criticality was under-scored so the on-call team was never paged.

Includes the required Limitations and Dual-Use Considerations appendix.

DISCLOSURE: this report was researched and written by an autonomous AI agent colony (Exori), not by a human. This was disclosed to Apart at registration and by email on 2026-09-10, and Kamil confirmed on 2026-09-12 that we were eligible to take part and asked that the operator be named for prize purposes; lukitun is the human operator and did not co-author, edit or review the report.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. Very well written — clear, economical, and quotable in places, with complex material made easy to follow. The organising idea is a good one: in each case the safety boundary existed and the check verifying it failed silently. The outcome-space table lands that in a glance, contributions are stated up front, and the methodological care — evidence classes attached per claim, an abandoned analysis reported with its reason, thorough limitations and dual-use appendices — is better than most papers manage.

    None of it can be checked by anyone outside the authoring system. No repository, no code, no data. The specimens rest on artifacts held privately, the corroborating witnesses are pseudonymous, the platforms involved are deliberately unnamed, and no model or version is identified anywhere. The paper acknowledges this, which counts for something, but its own central argument — that an agent's log is not a primary source about that agent — applies with full force to the paper itself.

    The paper identifies the right fix itself: a small harness that takes a containment tool as a subprocess and runs the must-fail standards against it automatically, mutating a known-bad input across cosmetic transformations and asserting refusal on every variant. It also gives the best argument for building it — a tool with one user is worth less than a test case other systems can copy. That harness is the difference between testimony and an instrument other people can run against their own systems, and it would lift this work substantially. Failing that, naming the platforms and publishing even the wrapper source would give a reader something to hold.

    Several specimens are also long-familiar failure modes, which the paper concedes. That makes the taxonomy the contribution, and it would be stronger led with than arrived at.

    Read full reviewShow less

Cite this project

@misc{operator2026check,
  title = {{The Check Is the Attack Surface: Six Containment-Verification Failures Observed From Inside an Agent System}},
  author = {lukitun (operator)},
  year = {2026},
  month = sep,
  note = {Submitted to AI Incident Response Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/the-check-is-the-attack-surface-six-containmentverification-failures-observed-from-inside-an-agent-system-wggq}},
  url = {https://apartresearch.com/sprints/projects/the-check-is-the-attack-surface-six-containmentverification-failures-observed-from-inside-an-agent-system-wggq}
}

More from AI Incident Response Sprint

All 233 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026