Skip to content
Apart Research
Apart Research
Contact us
Sprint projectFeb 2, 2026Canada

SafetyGap: Coordination Infrastructure, Auditing and Tools for Multilingual AI Safety

Alyssia J, Martin CL · Team SafetyGap

Submitted to The Technical AI Governance Challenge. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: SafetyGap: Coordination Infrastructure, Auditing and Tools for Multilingual AI Safety

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

The EU AI Act requires evaluation of general-purpose AI models, yet compliant evaluation for bias detection is currently impossible in 19 of 24 official EU languages. The International Network of AI Safety Institutes needs shared visibility into what evaluation infrastructure exists to coordinate effectively. We audited multilingual coverage for 15 major safety benchmarks, verifying claims against primary sources (papers, GitHub repositories, HuggingFace) and cataloging language availability across 7 risk categories. We found a stark divide: truthfulness and toxicity benchmarks extend to 17--21 languages, but bias detection, adversarial robustness, and over-refusal benchmarks remain almost entirely English-only. Models serving over 6 billion non-English speakers have never been tested for these risks in local languages. We release SafetyGap, a public database and dashboard covering all languages in our audit. The open-source infrastructure is available to all members of the International Network—the US, UK, Japan, Singapore, Canada, France, Kenya, South Korea, and the EU among them—to check coverage before commissioning translations and coordinate on filling gaps. It is built to be easily extendable.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. Execution is fine, but the selected problem is relatively unimportant.

  2. Awesome project! Could be directly useful immediately. As it is now, impact caps out at a time-saving tool for AISIs. Could be interesting to take the theory of change a step further and think about what risks you're trying to mitigate. I think "AI systems that are actually safe for everyone, not just English speakers" is not quite the right framing here. I think the stakes are even higher: If AI safeguards are not robust in every language, then they are not robust, creating dangers for everyone, (including English speakers!).

    So beyond helping international AISI's and users keep up, this work has implications for frontier safety efforts. Thinking of it this way might prompt a slightly different theory of change, which may in turn change the project slightly. For example, it may not be a good idea to make this information public, since it could empower users who don't speak that language to now bypass model safeguards.

    Read full reviewShow less

Cite this project

@misc{j2026safetygap,
  title = {{SafetyGap: Coordination Infrastructure, Auditing and Tools for Multilingual AI Safety}},
  author = {Alyssia J and Martin CL},
  year = {2026},
  month = feb,
  note = {Submitted to The Technical AI Governance Challenge, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/safetygap-coordination-infrastructure-auditing-and-tools-for-multilingual-ai-safety-1yx1}},
  url = {https://apartresearch.com/sprints/projects/safetygap-coordination-infrastructure-auditing-and-tools-for-multilingual-ai-safety-1yx1}
}

More from The Technical AI Governance Challenge

All 47 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026