Skip to content
Apart Research
Apart Research
Contact us
Sprint projectMay 25, 2026Ithaca

Zero-DoF Spec-Conditioned Decoding

Arya Datla, Prajwal Reddy · Team Corn Farmers

Submitted to The Secure Program Synthesis Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Zero-DoF Spec-Conditioned Decoding

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

While Large Language Models (LLMs) excel at code generation, their open-ended optimization for token likelihood over mathematical correctness introduces a severe security liability: excessive generative degrees of freedom. This structural flaw embeds subtle semantic vulnerabilities into functional software, while current post-hoc generate-then-test repair loops suffer from steep token latency and context-fragmenting patch slop. To resolve this, we present the Zero-DoF Spec-Conditioned Decoding Engine (Zero-SCD). The core novelty of Zero-SCD lies in shifting the verification burden directly into the autoregressive decoding pipeline. By leveraging an incremental Semantic AST Parser, the engine groups streaming tokens into complete, executable statements, executing them on the fly within a secure, sandboxed runtime against static deny-lists over 15 API families and sandboxed predicate checks. Empirical evaluation on a 146-case benchmark demonstrates the exact operational boundaries and pipeline stability of the system. In direct oracle tracking, the prototype successfully blocked 100% of curated unsafe code snippets, with rejections driven by restricted imports, restricted calls, and a 50ms halting timeout, yielding a 95% confidence interval for the block rate of [0.93, 1.00]. Conversely, a 94-prompt end-to-end sweep demonstrates robust pipeline stability under a deterministic stub-model regime, with all runs safely accepted. These results show that while true adversarial robustness under open-ended generation requires future live model testing, inline executable oracles successfully implement highly reproducible rejection policies at critical safety decision points.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. This project proposes a useful direction for secure program synthesis: move safety checks into the generation loop rather than waiting until after the full program has been produced. The idea of semantic checkpoints, sandboxed execution, rollback, and diagnostic feedback is relevant and potentially impactful. If made robust with real LLM generation, this could reduce wasted repair loops and catch unsafe code earlier.

    The main strength is the system framing. The project correctly identifies that syntax-level constrained decoding is not enough for secure code, because dangerous programs can be syntactically valid while violating semantic or runtime safety constraints. Integrating executable oracles into decoding is a promising approach, and the report is honest that the current implementation mainly demonstrates reproducible rejection policies rather than full adversarial robustness.

    The main limitation is the evaluation. The oracle blocks 52/52 curated unsafe snippets, which is a useful unit-test result, but the end-to-end prompt sweep uses a deterministic stub model that always emits benign code. That means the experiment does not yet test the core claim: whether Zero-SCD improves safety during real LLM code generation. The unsafe prompt labels are also not meaningful if the generated code is always safe. As written, the results show that the sandbox and deny-list work on a fixed negative set, not that the decoding method works under adversarial or realistic model behavior.

    To strengthen the project, I would prioritize running the same pipeline with an actual LLM, logging generated code, and comparing against an unguarded baseline. The evaluation should report false positives on benign code, false negatives on adversarial generated code, and the latency/token cost of rollback. It would also help to separate simple deny-list blocking from deeper semantic property checking, since restricted imports and calls are useful but do not fully demonstrate specification-conditioned decoding.

    Overall, this is a promising hackathon prototype with a good security intuition, but the current evidence is preliminary. The architecture is interesting; the next version needs a real-model evaluation to show that inline semantic checking changes generation outcomes rather than only validating fixed snippets.

    Read full reviewShow less

  2. Disparity in initial idea/claims and implemented methodology. Would also be useful to add more number to back the paper.

Cite this project

@misc{datla2026zerodof,
  title = {{Zero-DoF Spec-Conditioned Decoding}},
  author = {Arya Datla and Prajwal Reddy},
  year = {2026},
  month = may,
  note = {Submitted to The Secure Program Synthesis Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/zerodof-specconditioned-decoding-5di0}},
  url = {https://apartresearch.com/sprints/projects/zerodof-specconditioned-decoding-5di0}
}

More from The Secure Program Synthesis Hackathon

All 61 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026