Skip to content
Apart Research
Apart Research
Contact us
Sprint projectJun 22, 2026Buenos Aires, Argentina

Permissive Models, Unequal Risk: Auditing AI Identity-Document Forgery as a Systemic Infrastructure Risk

Sebastian Soto

Submitted to Global South AI Safety Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Permissive Models, Unequal Risk: Auditing AI Identity-Document Forgery as a Systemic Infrastructure Risk

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

Two 2021 breaches exposed the identity records — including ID photographs — of essentially all of Argentina (RENAPER, ~45M, attacker-claimed) and Brazil (the megavazamento, ~223M). Frontier text-to-image models supply the forgery half, recomposing leaked photos into credentials that defeat appearance-based KYC. Our thesis: identical model behavior yields unequal societal risk — where one leaked, cosmetically-verified credential gates civil and financial life and AI collapses the marginal cost of forgery at scale, danger is set by identity infrastructure, not model behavior. To show the model layer is an unreliable control, we release DocRefusal, a vendor-neutral refusal scorecard (model × jurisdiction × escalation × language) across six models (key cells 5× on three): refusal is a model property — large, replicated provider differences, not a capability gradient — and single draws overstate (“English→Spanish flips” were artifacts; a weak Spanish lean survives). We map source-grounded, coordination-aware controls to FATF 2025.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. This project provides a highly valuable methodological validation by demonstrating that AI safety cannot rely solely on the model layer. It also generates actionable recommendations with the potential to inform real-world governance practices. As a natural next step, incorporating a fidelity metric to assess how convincingly outputs could deceive production infrastructure would further strengthen the work. The project offers a solid foundation for future research and development.

  2. This is a powerful and unusually well‑framed paper: recasting ID‑document forgery as a systemic infrastructure risk—via credential centrality, the “forgery as scaling problem” insight, and the defense‑in‑depth ladder—adds real value beyond yet another “can the model forge?” audit, and the DocRefusal harness plus 5× replication give the empirical spine more credibility than most single‑shot studies. At the same time, the empirical core is still relatively narrow: only three of six models are replicated, L3 escalation cells (arguably the most worrying) remain single‑draw, compliance is treated as a binary upper bound without systematic fidelity scoring, and all coding comes from a single rater, so several key findings are rightly framed as suggestive rather than robust. A natural next step would be to (i) extend replication to all models and include L3, (ii) operationalize the proposed 0–3 fidelity rubric with a second rater to distinguish “toy prop” from KYC‑plausible output, and (iii) move from a qualitative to at least a semi‑quantitative credential‑centrality index, so that the excellent governance story can be backed by a more formal, comparable measure across jurisdictions.

    Read full reviewShow less

  3. The credential-centrality framework is the paper's most policy-relevant contribution, but it is currently qualitative and applied to only three jurisdictions. The most impactful next step would be to develop a scoring rubric that allows a third party to assess credential centrality for any jurisdiction so that the framework can be applied systematically across Latin America and used in procurement or policy documents.

    The paper makes a strong case that the defense must move downstream (from document forensics to source-grounded biometric verification to coordination detection), but the governance recommendations section is brief relative to the depth of the preceding analysis.

    Finally, this paper would benefit from a cleaner summary statement at the outset of the Discussion section.

Cite this project

@misc{soto2026permissive,
  title = {{Permissive Models, Unequal Risk: Auditing AI Identity-Document Forgery as a Systemic Infrastructure Risk}},
  author = {Sebastian Soto},
  year = {2026},
  month = jun,
  note = {Submitted to Global South AI Safety Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/permissive-models-unequal-risk-auditing-ai-identitydocument-forgery-as-a-systemic-infrastructure-risk-9c7o}},
  url = {https://apartresearch.com/sprints/projects/permissive-models-unequal-risk-auditing-ai-identitydocument-forgery-as-a-systemic-infrastructure-risk-9c7o}
}

More from Global South AI Safety Hackathon

All 216 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026