Misinformational AI-Generated Academic Papers
Aaron Sandoval, Akash Kundu, Layla Adam · Team The Fake Academics
Submitted to AI capabilities and risks demo-jam. Sprint projects are early-stage work by participants, not Apart Research publications.
This study explores the potential for generative AI to produce convincing fake research papers, highlighting the growing threat of AI-generated misinformation. We demonstrate a semi-automated pipeline using large language models (LLMs) and image generation tools to create academic-style papers from simple text prompts.
Reviews
Very cool idea! This is solid progress towards auto-generation of research papers. I particularly liked (1) including figures & (2) starting with a many similar papers as a baseline.I think it would’ve been nice to include some research papers that you consider problematic as examples.
Good idea!I feel like it would have been better to generate a paper proving a point of view generated by the user, as the frontend only allows downloading the generated paper, which doesn’t prove automation.
Nice work! Showing rendered LaTeX is a nice direct way to demonstrate LLMs’ ability to generate it. For next steps, I’d be interested to see a more interactive version of this demo, and more support for the user in understanding where the generated paper is or isn’t plausible.
Cite this project
@misc{sandoval2024misinformational,
title = {{Misinformational AI-Generated Academic Papers}},
author = {Aaron Sandoval and Akash Kundu and Layla Adam},
year = {2024},
month = aug,
note = {Submitted to AI capabilities and risks demo-jam, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/misinformational-ai-generated-academic-papers}},
url = {https://apartresearch.com/sprints/projects/misinformational-ai-generated-academic-papers}
}More from AI capabilities and risks demo-jam
- 1st place by peer reviewView project: Speculative Consequences of A.I. Misuse
Speculative Consequences of A.I. Misuse
Team S.C.A.M.
This project uses A.I. Technology to spoof an influential online figure, Mr Beast, and use him to promote a fake scam website we created.
- View project: Demonstrating LLM Code Injection Via Compromised Agent Tool
Demonstrating LLM Code Injection Via Compromised Agent Tool
This project demonstrates the vulnerability of AI-generated code to injection attacks by using a compromised multi-agent tool that generates Svelte code. The tool shows how malicious code can be injected during the code …
- View project: Phish Tycoon: phishing using voice cloning
Phish Tycoon: phishing using voice cloning
Phish Tycoon
This project is a public service announcement highlighting the risks of voice cloning, an AI technology capable of creating synthetic voices nearly indistinguishable from real ones. The demo involves recording a user's …