MAIR: The Misaligned AI Incident Reporting Standard

This is an ambitious effort that seeks to provide a complete solution for incident reporting as part of a very rapid sprint; the fact that it falls short in various ways is a function of overambitious goals, and the project itself shows promise.

The literature review and related work is unfortunately very incomplete, partly because so many of the other related projects are still actively under development, and semi-public. (e.g. ISO AWI 25870 is a non-public working draft; https://www.iso.org/standard/91804.html , and the NIST workshop has evidently not yet led to a publication: https://www.nist.gov/news-events/events/2026/05/nist-workshop-ai-incident-management .) However, this means that many of the otherwise novel contributions here are already being discussed within groups discussing the issue.

The contribution also proposes a new set of dimensions that are not validated, understandably given the sprint length, and the rater variance used sensitivity instead of more standard and more rigorous inter-rater reliability measures. Proposing that this is the right way forward moves to far, as does saying that this "fixes this problem" on the basis of a single set of events. The overclaiming is actively unhelpful, and undermines the very useful contribution.

In summary, the work is excellent, but the ambition of claiming this as a new standard is unfortunate, as it would be far more useful and impactful to position it as an alternative metric useful for reporting frameworks being developed.

This work proposes MAIR, a framework for rating the severity of AI incidents similarly to CVSS for vulnerabilities.

The proposal is interesting and clearly presented, but its novelty is unclear given existing frameworks such as the OECD Common Reporting Framework for AI Incidents and OWASP AIVSS.

More importantly, the paper claims that MAIR can identify when legal reporting requirements are triggered, but does not explain how its scores and thresholds are derived from the cited regulations or provide legal validation of that mapping.

MAIR is a useful attempt to give AI incidents a more structured language than free form post mortems, and the dashboard makes the proposal easy to understand. I like separating egress, detection and intent diff.

My main concern is validation. Applying the framework to 30 incidents shows that it can be used, but not yet that its ordering or thresholds are correct. I’d next use multiple blinded raters, measure agreement, validate the weights on held-out incidents and keep the legal-reporting mapping separate from the severity score itself.