Limitations in existing regulatory mechanisms when applied to attacks by AI agents

This project examines the recent incidents against the incident reporting criteria for the EU AI Office and California's SB53 to assess whether they fall within scope of reportable incidents and propose modifications to these criteria. This is a useful activity which could be of value to these regulators and other policy makers considering putting in place similar reporting requirements.

I found the paper clear, read naturally and had a good level of detail. There were some decisions made in the paper around which dimensions were important which would have benefited from being better defended:

Why do third parties need to be able to report incidents, given the legal obligations fall on the AI developers and they would have the best access to the details surrounding the incident? The examples you give of the wiki’s administrator and a third party with access gave enough to imply reasoning here, but this could have been made more explicit and backed up with data.

e.g. Evidence points to both incidents being first identified by the compromised organisations (Huggingface and DSEwiki) - if they suspected an AI related attack but could not identify the source or did not get a response back from the developer, this supports the argument for these being able to report.

Also, if a third party such as METR observe a developer withholding information, can they report? Perhaps under whistleblowing provisions.

Given only the DSEwiki was reported to the EU AI Office, the jurisdictional scope of the incident was likely a scope determinant. Do you see this as a problem? That the regulator would then have to rely on public reporting sources. Even if not impacting directly the relevant jurisdiction, should incidents which inform new dangerous capabilities be reportable? This could point to a clearer gap what is enforceable with patchwork regulations too.

Challenges could discuss false reports and verification.

This submission has set out to build a tool to help victims of agentic attacks file reports. It found there was nowhere to file. The authors observe that SB 53 exempts deceptive model behaviour occurring during an evaluation designed to elicit it, which is the setting the Hugging Face incident arose in. Additionally, the authors did not stop at their own reading of the law but checked it against the California agency's own position and against the Commission's refusal to say which mechanism OpenAI filed under.

However, the claim is broader than the evidence. Only the AI Act and SB 53 were examined, so what the paper can support is that no mapped AI-specific channel takes third-party reports. A compromised wiki administrator (a role the paper uses) still has breach notification, national incident channels and ordinary computer-misuse law, and the paper needs to explicitly say why those do not fill the gap. Article 73 of the EU AIA goes undiscussed. Second, the three failure points (threshold, who may file, no public record) are used as a framework but these are not justified.