AI Alignment Knowledge Graph

I like the idea of an alignment knowledge graph! On top of directly helping researchers to navigate the complex and interdisciplinary landscape of alignment research, progress on the core knowledge graph can then later be used by other tools for research augmentation.

• Polish of tool is impressive
• Good to see the code works and has a README/license etc

‍

Cool project! LLMs are definitely more costly and take up a lot of processing time, but can lead to better clustering than embeddings. I like the hierarchy and backlinks. It reminds me of an updated version of FLI’s mindmap: https://futureoflife.org/valuealignmentmap/ except it is LLM generated. The visualization is great, but my main concern for these kinds of things is that they look cool and are fun to play with, but don’t lead to constant use because it doesn’t feel optimal for learning. I think using the backlinks, summarizations, pointing to the papers, and such, are a good start for making this kind of thing more practical. I think the adoption issue is partially that the entire map feels overwhelming. It’s hard to decide where to start and there are too many possible directions.I think what could be added would be a chat interface where you can actually ask an LLM specific questions about a subset of papers and adding curated open questions to give new researchers direction for what kind of questions they should have in mind and how a subfield relates to the overall agenda for alignment.One thing that could potentially be interesting is related to my idea of “the alignment mosaic”, where we carefully curate questions we need to answer in alignment as well as different agendas. Then, we populate that foundation with insights and related work from the literature using the pipeline you’ve built. That way, we start from clusters of questions we need to resolve in alignment and the user can see things like critiques, related work, projects they could pursue next, etc.