minTranscoders
Bronson Schoen
Submitted to ARENA 4.0 Interpretability Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
Attempting to be a minGPT like implementation for transcoders for MLP hidden state in transformers - part of ARENA 4.0 Interpretability Hackathon via Apart Research
Reviews
No public critique yet.
Cite this project
@misc{schoen2024mintranscoders,
title = {{minTranscoders}},
author = {Bronson Schoen},
year = {2024},
month = sep,
note = {Submitted to ARENA 4.0 Interpretability Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/mintranscoders}},
url = {https://apartresearch.com/sprints/projects/mintranscoders}
}More from ARENA 4.0 Interpretability Hackathon
- View project: Interpreting a toy model for finding the maximum element in a list
Interpreting a toy model for finding the maximum element in a list
Interpreting a toy model for finding the maximum element in a list
- View project: nnsight transparent debugging
nnsight transparent debugging
We started this project with the intent of identifying a specific issue with nnsight debugging and submitting a pull request to fix it. We found a minimal test case where an IndexError within a nnsight run wasn’t …
- View project: Latent Space Clustering and Summarization
Latent Space Clustering and Summarization
I wanted to see how modern dimensionality reduction and clustering approaches can support visualization and interpretation of LLM latent spaces. I explored a number of different approaches and algoriths, but ultimately …