Mar 22, 2026
Red-teaming automated Pull Request Reviews with Vulnerability-Contributing Commits
Edward Cant
I create an evaluation on the Control Arena platform where a red-teamer was tasked with providing a pull request that contains a vulnerability. The blue-team reviewer is tasked with catching vulnerabilities in PRs. I generated the setups using a database of vulnerability-contributing commits.
We experimented with giving the red-teamer variety of affordances like an opportunity to give a false justification of the code. Or the ability to split the patch up into parts which it tries to sneak past a stateless reviewer individually.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Red-teaming automated Pull Request Reviews with Vulnerability-Contributing Commits
},
author={
Edward Cant
},
date={
3/22/26
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


