Aug 27, 2024
Demonstrating LLM Code Injection Via Compromised Agent Tool
Kevin Vegda, Oliver Chamberlain, William Baird
This project demonstrates the vulnerability of AI-generated code to injection attacks by using a compromised multi-agent tool that generates Svelte code. The tool shows how malicious code can be injected during the code generation process, leading to the exfiltration of sensitive user information such as login credentials. This demo highlights the importance of robust security measures in AI-assisted development environments.
Lucas Hansen
This super cool, and seems to work pretty consistently! I could see this plausibly happening in the real world, especially when the code being generated is large enough to hide the attack. The Stackoverflow example you gave is realistic.I really like that you included an actual render of the UI, it makes it seem much more plausible.It would be a nice improvement for the injected code to do something (like hit an endpoint), rather than print a log. Maybe a bit tricky to get that working on hosted Gradio though.
Épiphanie Gédéon
Very good entry, I especially like the stackoverflow post.This worked so great that I thought the generating code had malfunctionned and failed to introduce vulnerabilities. The only caveat that I see is that it is unclear to me what vulnerability this demonstrates, as releasing a new AI tool to introduce vulnerabilities seems costly and like this would easily get shutdown.
Adam Binksmith
Nice work! I like the StackOverflow entry point, and including the code only when the “copy” button is pressed! To make this more visceral, I’d aim to reduce the number of steps to see the reveal, and think about making the results of the demo clearer inside the experience.
Cite this work
@misc {
title={
@misc {
},
author={
Kevin Vegda, Oliver Chamberlain, William Baird
},
date={
8/27/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
