RAFT: Gradual Typing, Invariance Enforcement, and Property Verification in Research Python.
Thomas Winninger, Antonin Peronnet
AI researchers prototype in short scripts or Jupyter notebooks and push directly to state-of-the-art frameworks like `transformers` or `vLLM`. They lack the time for rigorous testing or manual verification. Moreover, the nature of their work makes traditional test or spec-driven development difficult to apply. We propose an automated, iterative pipeline to bridge the gap between quick experimental scripts and shareable, verifiable code, utilizing AI and strict static analysis to gradually introduce typing, contracts, and property verification. We validate our results with a backdoor detection experiment. A small reviewer `gemma4 e4b` is more efficient at finding bugs and backdoors after the application of our method. Going from 46% recall on vanilla code, to 95% detection on the `transformers`'s library.
The paper highlights a real problem which is turning ad‑hoc research code into verifiable artifacts and the ablation study is thoughtfully designed. That being said, the core contribution feels limited, as RAFT mainly orchestrates existing static analysis tools rather than introducing new techniques. The large reported gain in backdoor detection appears driven mostly by a prompting change, not by code transformation itself. Evaluating on only seven repositories with a small, quantized model limits confidence in the results; testing on larger codebases and stronger models would make the claims more convincing.
Gradual typing is a strong idea for secure program synthesis -- it allows rapid code development to be moved towards guarantees. This sprint covers two directions: a tool chain called rafty, built of various gradual typing related tools, and a new tool called annassert, which converts python asserts into type annotations that can then be checked by other tools as compile time. Going forward I'd be curious about how much typical python code is in the format that annassert suggests; this could be checked by running over the samples in the repo or more broadly some chunk of repos from github.
Cite this work
@misc {
title={
(HckPrj) RAFT: Gradual Typing, Invariance Enforcement, and Property Verification in Research Python.
},
author={
Thomas Winninger, Antonin Peronnet
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


