Jul 29, 2024
Grant Application Simulator
Michaël Trazzi
We build a VS Code extension to get feedback on AI Alignment research grant proposals by simulating critiques from prominent AI Alignment researchers and grantmakers.
Simulations are performed by passing system prompts to Claude 3.5 Sonnet that correspond to each researcher and grantmaker, based on some new grantmaking and alignment research methodology dataset we created, alongside a prompt corresponding to the grant proposal.
Results suggest that simulated grantmaking critiques are predictive of sentiment expressed by grantmakers on Manifund.
Natalia Pérez-Campanero Antolín
Given how obscure the grant evaluation process can be and how much of an impact grants can have in facilitating AIS research, this is a fantastic and much needed tool. The tool is well built, with solid and well explained methodology. Would be particularly interested to see how this can be incorporated into different project stages. Well done!
Marc Carauleanu
Great project! I appreciate that you curated a dataset and that you integrated it with LLMs into a tool that is specifically useful for alignment researchers in such a short time! I would love to see further iterations on the user interface, as well as additional features that make parsing critiques/debates easier. This tool can be useful for red-teaming your alignment proposal as well as brainstorming, as the data obtained from mentor comments is quite valuable for priming the model correctly.
Jonny Spicer
Given the frequent comments about lack of AI safety researchers relative to those working on capabilities, I think this tool would be an excellent way to improve quality of grants and therefore increase the number of researchers able to make progress on various safety agendas. I think your methodology is solid and you clearly have a deep understanding of how to engineer prompts effectively. As an additional next step, I’d love to see this be a tool that accepts Google Docs, as I assume this is where the majority of grant proposals gets written.
Jacques Thibodeau
Love this! It seems especially useful for trying to raise the bar for projects and equalizing grant applications (making it less likely for a project to get accepted just because one person has some special insight for making a good application). This has many of the features for making it generally useful for producing better research agendas, better theories of change, prioritizing which project to pursue, and raising the bar for all research projects (for example, getting people to collaborate or get a mentor).
Cite this work
@misc {
title={
@misc {
},
author={
Michaël Trazzi
},
date={
7/29/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
