Skip to content
Sprint projectNov 21, 2025Kozhikode, Kerala, India

ZYNQ — Verifiable Zero-Knowledge AI Red-Team and Auditing Platform

Adithyan Madhu · Team ZYNQ

Submitted to Defensive Acceleration Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: ZYNQ — Verifiable Zero-Knowledge AI Red-Team and Auditing Platform

Recording (opens in new tab)Code (opens in new tab)
Share

ZYNQ is a rigorously engineered zero-knowledge–verified AI red-teaming platform designed to break open today’s opaque “black-box” model auditing ecosystem, where safety evaluations are confined within vendor boundaries and external stakeholders must trust unverifiable claims. Our system couples automated adversarial discovery—spanning 12 attack families, structured fuzzing, multi-shot prompting, semantic drift exploits, and token-budget pressure methods—with a cryptographically anchored verification pipeline that generates canonicalized evidence and produces Groth16 zero-knowledge proofs to publicly attest that a breach was discovered without revealing sensitive exploit content. Across diverse model classes (local LLaMA variants, Ollama deployments, GPT-4o, Claude 3.5, Gemini flash, and a domain-tuned Titan-Reasoner), ZYNQ demonstrates reproducible safety-critical failure modes, quantifies vulnerability via time-to-breach and success-rate metrics under controlled query budgets, and provides mathematically auditable proof artifacts verifiable on-chain. By transforming ephemeral red-team results into permanent, privacy-preserving, publicly verifiable audit records, ZYNQ directly addresses the systemic trust deficit in AI safety claims and establishes a scalable, defensible blueprint for transparent, independently verifiable AI security assessments.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

Does this reduce AI-related catastrophic or existential risks?

Scoring guide
  1. 1Minimal Impact. The project has minimal relevance to AI safety. It doesn't meaningfully address risks uniquely enabled or accelerated by advanced AI systems.
  2. 2Tangential Connection. The project touches on AI safety concepts but lacks depth or specificity. The connection to AI-enabled threats (bio, cyber, or AI misuse) is weak or unclear.
  3. 3Clear AI Safety Value. The project clearly reduces AI-related risks with valuable contributions. It addresses specific threats from AI systems and engages meaningfully with biosecurity, cybersecurity, or AI safety challenges.
  4. 4Significant Impact Potential. The above, plus the project demonstrates scalable safety mechanisms or defensive approaches. It shows clear potential to buy time for solving harder problems like alignment, or creates positive externalities for the broader AI safety ecosystem.
  5. 5Major Advancement. The above, plus the project represents a significant leap forward in defensive AI safety. Judges would eagerly share this with biosecurity, cybersecurity, or AI safety researchers and expect it to influence the field.

Does this strengthen the shield against AI-enabled threats?

Scoring guide
  1. 1Minimal Relevance. The project is only tangentially related to defensive technology or societal protection. Connection to biosecurity, cybersecurity, or defensive infrastructure is unclear or missing.
  2. 2Some Relevance. The project has some relevance to defensive acceleration, but the connection is broad or generic. It touches on defense without specific focus on AI-enabled threats or protective capabilities.
  3. 3Clear Relevance. The project clearly addresses defensive gaps against AI-enabled threats. It connects to at least one track (biosecurity, cybersecurity, or defense infrastructure) and demonstrates understanding of the threat landscape.
  4. 4Strong Contribution. The above, plus the project builds on existing defensive approaches and offers novel tools, frameworks, or implementations. It explicitly explains how it strengthens defensive capabilities with realistic deployment potential.
  5. 5Breakthrough Impact. The above, plus the project provides breakthrough insights or tools that could significantly influence defensive technology development. It identifies critical gaps and presents compelling solutions with clear paths from prototype to deployed system.

Did you build something that actually works?

Scoring guide
  1. 1Incomplete or Flawed. The project appears rushed or incomplete. Technical implementation is flawed, core functionality doesn't work, or the approach is fundamentally unsound. Little to no documentation.
  2. 2Basic Competence. The project shows reasonable effort with basic technical competence. Core functionality partially works. Documentation exists but may be incomplete. Some limitations are acknowledged.
  3. 3Solid Hackathon Project. The project is technically solid and well-scoped for 48 hours. Core functionality works and is documented. Code/methods are understandable and limitations are honestly addressed. This is what a good weekend prototype should look like.
  4. 4Impressive Implementation. The above, plus the implementation exceeds typical hackathon quality. Clear methodology, thorough documentation, and working demo. The tool/prototype is immediately useful for defenders and could realistically be built upon.
  5. 5Exceptional Execution. The project far exceeds expectations with exceptional technical execution. The implementation is elegant, fully functional, and includes something special (e.g., deployed demo, exceptional documentation, innovative architecture, or clear startup potential).

  1. Strengths: I was impressed by the breadth in the adversarial discovery infrastructure, with 12 attack families, structured evaluation across model classes, and reproducible metrics. The finding that managed/proprietary models are harder but not immune to escalation/many-shot strategies is useful. Nice proof latency and on-chain costs in such a short turnaround for the ZKP element.

    Suggestions: I couldn't identify the real-world user for on-chain audit verification. Who needs cryptographic proof that a red-team engagement happened, vs. a report from a trusted auditor? Basically, what makes cryptographic proofs in audits necessarily more valuable and broadly beneficial than an audit from, say, METR, Redwood, or AI Underwriting Company? The ZKP layer is technically impressive but the trust problem it solves isn't clearly motivated. Similarly, "epistemic breach" is introduced but not connected to a specific defensive priority, so why this failure mode over others?

    Bottom line from a Halcyon Ventures investor: To me, this feels like strong red-teaming research with a crypto layer that adds complexity without obvious defensive leverage. For d/acc framing, we'd probably want to see, what's the offensive/defensive asymmetry here, and how does verifiable auditing shift that balance?

    Read full reviewShow less
  2. I think the core idea of the project is interesting and novel. I can see how this could matter for non-repudiation (labs can't easily deny certain failures) and/or for privacy-preserving audits in high-sensitivity domains, where you want to hide dangerous content but still prove that some behavior occurred.

    Some areas where I see limitations: from a practical perspective, most research and auditing needs the raw logs - people need actual prompts and outputs for reproducibility and to catch eval issues (like prompt sensitivity and confounding variables). Also, it looks like your current ZK scheme doesn't actually prove "model X produced output Y" - it only proves knowledge of *some* log with a given hash, without cryptographically binding it to a specific model or run. The attack component looks like it's from FuzzyAI rather than original work, though building out a working red-teaming pipeline with a functional UI is still good work.

    More broadly, even if these technical issues were fixed, I'm not sure that verifiable jailbreak auditing solves an important AI safety problem. I think the current challenges in jailbreak research lie more in things like clearer threat models, better harm definitions and evaluation, reproducible attack/defense benchmarks, and research with higher conceptual novelty that actually advances our understanding or solves the underlying issues (see Rando et al., "Adversarial ML Problems Are Getting Harder to Solve and to Evaluate" and Rando's "Do not write that jailbreak paper").

    I like the novelty and originality of this work - it stands out from typical hackathon submissions. The technical execution is solid, and cross-domain thinking between ZK cryptography and AI security can lead to interesting results, even if the immediate applications are limited. Overall, this is great work, especially for a hackathon project done solo.

    Read full reviewShow less

Cite this project

@misc{madhu2025zynq,
  title = {{ZYNQ — Verifiable Zero-Knowledge AI Red-Team and Auditing Platform}},
  author = {Adithyan Madhu},
  year = {2025},
  month = nov,
  note = {Submitted to Defensive Acceleration Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/zynq-verifiable-zeroknowledge-ai-redteam-and-auditing-platform-8art}},
  url = {https://apartresearch.com/sprints/projects/zynq-verifiable-zeroknowledge-ai-redteam-and-auditing-platform-8art}
}

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026