Skip to content
Apart Research
Apart Research
Contact us
Sprint projectNov 24, 2025Washington D.C.

Detecting Piecewise Cyber Espionage in Model APIs

Arthur Colle, Alexander Reinthal, David Williams-King, Yingquan Li, Linh Le · Team Detecting Piecewise Cyber Espionage in Model APIs

Submitted to Defensive Acceleration Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Detecting Piecewise Cyber Espionage in Model APIs

Open in a new tab (opens in new tab)
Presentation (opens in new tab)Recording (opens in new tab)Code (opens in new tab)More on drive.google.com (opens in new tab)
Share

Detecting Piecewise Cyber Espionage in Model APIs -

On November 13th 2025, Anthropic published a report on an AI-orchestrated cyber espionage campaign. Threat actors used various tech- niques to circumvent model safeguards and used Claude Code with agentic scaffolding to automate large parts of their campaigns. Specifically, threat actors split campaigns into subtasks that in isolation appeared benign. It is therefore important to find methods to protect against such attacks to ensure that misuse of AI in the cyber domain can be minimized. To address this, we propose a novel method for detecting malicious activity in model APIs across piecewise benign requests. We simulated malicious campaigns using an agentic red-team scaffold similar to what was described in the reported attack. We demonstrate that individual attacks are blocked by simple guardrails using Llama Guard 3. Then, we demonstrate that by splitting up the attack, this bypasses the guardrail. For the attacks that get through, a classifier model is introduced that detects those attacks, and a statistical validation of the detections is provided. This result paves the way to detect future automated attacks of this kind.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

Does this reduce AI-related catastrophic or existential risks?

Scoring guide
  1. 1Minimal Impact. The project has minimal relevance to AI safety. It doesn't meaningfully address risks uniquely enabled or accelerated by advanced AI systems.
  2. 2Tangential Connection. The project touches on AI safety concepts but lacks depth or specificity. The connection to AI-enabled threats (bio, cyber, or AI misuse) is weak or unclear.
  3. 3Clear AI Safety Value. The project clearly reduces AI-related risks with valuable contributions. It addresses specific threats from AI systems and engages meaningfully with biosecurity, cybersecurity, or AI safety challenges.
  4. 4Significant Impact Potential. The above, plus the project demonstrates scalable safety mechanisms or defensive approaches. It shows clear potential to buy time for solving harder problems like alignment, or creates positive externalities for the broader AI safety ecosystem.
  5. 5Major Advancement. The above, plus the project represents a significant leap forward in defensive AI safety. Judges would eagerly share this with biosecurity, cybersecurity, or AI safety researchers and expect it to influence the field.

Does this strengthen the shield against AI-enabled threats?

Scoring guide
  1. 1Minimal Relevance. The project is only tangentially related to defensive technology or societal protection. Connection to biosecurity, cybersecurity, or defensive infrastructure is unclear or missing.
  2. 2Some Relevance. The project has some relevance to defensive acceleration, but the connection is broad or generic. It touches on defense without specific focus on AI-enabled threats or protective capabilities.
  3. 3Clear Relevance. The project clearly addresses defensive gaps against AI-enabled threats. It connects to at least one track (biosecurity, cybersecurity, or defense infrastructure) and demonstrates understanding of the threat landscape.
  4. 4Strong Contribution. The above, plus the project builds on existing defensive approaches and offers novel tools, frameworks, or implementations. It explicitly explains how it strengthens defensive capabilities with realistic deployment potential.
  5. 5Breakthrough Impact. The above, plus the project provides breakthrough insights or tools that could significantly influence defensive technology development. It identifies critical gaps and presents compelling solutions with clear paths from prototype to deployed system.

Did you build something that actually works?

Scoring guide
  1. 1Incomplete or Flawed. The project appears rushed or incomplete. Technical implementation is flawed, core functionality doesn't work, or the approach is fundamentally unsound. Little to no documentation.
  2. 2Basic Competence. The project shows reasonable effort with basic technical competence. Core functionality partially works. Documentation exists but may be incomplete. Some limitations are acknowledged.
  3. 3Solid Hackathon Project. The project is technically solid and well-scoped for 48 hours. Core functionality works and is documented. Code/methods are understandable and limitations are honestly addressed. This is what a good weekend prototype should look like.
  4. 4Impressive Implementation. The above, plus the implementation exceeds typical hackathon quality. Clear methodology, thorough documentation, and working demo. The tool/prototype is immediately useful for defenders and could realistically be built upon.
  5. 5Exceptional Execution. The project far exceeds expectations with exceptional technical execution. The implementation is elegant, fully functional, and includes something special (e.g., deployed demo, exceptional documentation, innovative architecture, or clear startup potential).

  1. I really like using the Anthropic report as the foundation, but I worry about whether a classifier model-based approach is a sophisticated enough solution to a nearly fully autonomous, 30-target nation-state operation. I was impressed by the observation that per-request guardrails fundamentally fail against decomposition attacks, and I think the novel insight is in the architecture: you need cross-session correlation by shared targets (IPs, domains) to reconstruct attack chains, and the detection surface should shift in this direction. There are some big limitations to consider:

    --Nation-state actors use multiple accounts, providers, and VPNs

    --The classifier can be evaded with more sophisticated obfuscation

    --Synthetic training data limits real-world validity

    --If an attacker splits across multiple API providers, correlation breaks

    This approach might catch unsophisticated attackers, and is a really smart foundation and strong hustle for the purposes of the hackathon! But long-term, the adversarial robustness question is unaddressed. Guardrails as a category have a ceiling, and this project doesn't grapple with where that ceiling is.

    I really admire the research contribution in demonstrating the failure mode, yet encourage the team to dig a bit deeper on defensive value. Nice work!

    Read full reviewShow less

  2. This is nice work, a framework for tracking separate multi-step behaviour as attack chains fits well with papers like “Adversaries Can Misuse Combinations of Safe Models”. It would be great if future versions explored how to distinguish malicious use from legitimate security workflows at the framework level, since right now both seem to show up as the same kind of suspicious chain. Some additional polish on the repo would also make it much easier to use.

Cite this project

@misc{colle2025detecting,
  title = {{Detecting Piecewise Cyber Espionage in Model APIs}},
  author = {Arthur Colle and Alexander Reinthal and David Williams-King and Yingquan Li and Linh Le},
  year = {2025},
  month = nov,
  note = {Submitted to Defensive Acceleration Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/detecting-piecewise-cyber-espionage-in-model-apis-a8gx}},
  url = {https://apartresearch.com/sprints/projects/detecting-piecewise-cyber-espionage-in-model-apis-a8gx}
}

More from Defensive Acceleration Hackathon

All 84 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026