Skip to content
Sprint projectMay 24, 2026Durham ,NC

Lean checker validating Skill Plugin

Ivan Rojas · Team Ivan G. Rojas

Submitted to The Secure Program Synthesis Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Lean checker validating Skill Plugin

Code (opens in new tab)
Share

It goes through goals in lean and solves for a theorem. It asks sub questions about why a checker passed/failed and adds this as a topological layer (a chain complex). We estimate a betti table using qiskit for test problems of greater complexity and this estimates the time any checker takes for parsed questions in the homology.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. This project appears to wrap a prime_kernel_homology.ipynb pipeline into a reusable Lean checker validator, using persistent homology over embedded reasoning traces and a pregroup decision tree over several Lean kernel checkers. The idea of validating proof-checker behavior and reasoning traces is potentially relevant to secure program synthesis, but the submission does not yet make a clear case that the method solves a concrete safety problem.

    The main issue is clarity. The report uses many technical concepts - pregroup grammars, persistent homology, Betti numbers, QPE, torus barcodes, Lean checker kernels - but it is difficult to tell what is actually implemented, what is being evaluated, and what claim is supported by evidence. The test set appears to be only prime proofs for very small primes, and the report does not provide a clear success metric, baseline, failure case, or security-relevant result.

    The execution also seems thin for the stated ambition. The deliverables are mostly a plugin folder, a plan, a LaTeX record, and a signal-flow description. I could not identify a concrete validation result showing that this method catches incorrect proofs, distinguishes good from bad checkers, improves Lean verification, or prevents a secure synthesis failure mode. The QPE discussion is interesting but explicitly says there is no advantage at the current scale.

    To improve the project, I would strongly recommend simplifying the framing. Start with one clear task: for example, “given several Lean checker variants, detect which ones incorrectly accept invalid proofs.” Then provide a small benchmark with valid and invalid proofs, show which checkers pass/fail, and demonstrate that the validator catches a real discrepancy. The topology and quantum components should only be included if they directly improve that concrete validation task.

    Overall, this is imaginative, but currently too hard to interpret and not sufficiently grounded in evidence. The project needs a clearer problem statement, simpler evaluation, and a direct connection to secure program synthesis outcomes.

    Read full reviewShow less
  2. The paper results are written up in a way that makes it very challenging to decipher the purpose of this project. As an example, the abstract begins "We document a skill plugin that wraps the prime kernel homology.ipynb pipeline as a reusable Lean checker validator." prime_kernel_homology.ipynb is not a standard problem nor pipeline, so this sentence does very little to explain what is going on. The abstract continues and explains the MCP server "manages a pregroup-grammar decision tree" (which is never explained) over a set of primes 2,3, 5, and prospective extensions, 7, 11, 13. This without further context (which is not given in the rest of the paper), is incredibly challenging to decipher the larger impacts of.

Cite this project

@misc{rojas2026lean,
  title = {{Lean checker validating Skill Plugin}},
  author = {Ivan Rojas},
  year = {2026},
  month = may,
  note = {Submitted to The Secure Program Synthesis Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/lean-checker-validating-skill-plugin-5kfh}},
  url = {https://apartresearch.com/sprints/projects/lean-checker-validating-skill-plugin-5kfh}
}

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026