Interpreting a toy model for finding the maximum element in a list

We started this project with the intent of identifying a specific issue with nnsight debugging and submitting a pull request to fix it. We found a minimal test case where an IndexError within a nnsight run wasn’t correctly propagated to the user, making debugging difficult, and wrote up a proposal for some pull requests to fix it. However, after posting the proposal in the discord, we discovered this page in their GitHub (https://github.com/ndif-team/nnsight/blob/2f41eddb14bf3557e02b4322a759c90930250f51/NNsight_Walkthrough.ipynb#L801, ctrl-f “validate”) which addresses the problem. We replicated their solution here (https://colab.research.google.com/drive/1WZNeDQ2zXbP4i2bm7xgC0nhK_1h904RB?usp=sharing) and got a helpful stack trace for the error, including the error type and (several stack layers up) the line causing it.

Attempting to be a minGPT like implementation for transcoders for MLP hidden state in transformers - part of ARENA 4.0 Interpretability Hackathon via Apart Research

I wanted to see how modern dimensionality reduction and clustering approaches can support visualization and interpretation of LLM latent spaces. I explored a number of different approaches and algoriths, but ultimately converged on UMAP for dimensionality reduction and birch clustering to extract groups of tokens in the latent space of a layer.