RNNs represent belief state geometry in hidden state
Keenan Pepper · Team Keenan
Submitted to Computational Mechanics Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
The Shai et al. experiments on transformers (finding belief state geometry in the residual stream) have been replicated in RNNs with the hidden state instead of the residual stream. In general the belief state is stored linearly, but not in any particular layer, rather spread out across layers.
Reviews
No public critique yet.
Cite this project
@misc{pepper2024rnns,
title = {{RNNs represent belief state geometry in hidden state}},
author = {Keenan Pepper},
year = {2024},
month = jun,
note = {Submitted to Computational Mechanics Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/rnns-represent-belief-state-geometry-in-hidden-state}},
url = {https://apartresearch.com/sprints/projects/rnns-represent-belief-state-geometry-in-hidden-state}
}More from Computational Mechanics Hackathon
- View project: Unsupervised Recovery of Hidden Markov Models from Transformers with Evolutionary Algorithms
Unsupervised Recovery of Hidden Markov Models from Transformers with Evolutionary Algorithms
Stargazy Pie
Prior work finds that transformer neural networks trained to mimic the output of a Hidden Markov Model (HMM) embed the optimal Bayesian beliefs for the HMM's current state in their residual stream, which can be …
- View project: Looking forward to posterity: what past information is transferred to the future?
Looking forward to posterity: what past information is transferred to the future?
Nonkansa
I used mechanistic interpretability techniques to try to see what information the provided Random Randox XOR transformer looks at when making predictions by examining its attention heads manually. I find that earlier …
- View project: Investigating the Effect of Model Capacity Constraints on Belief State Representations
Investigating the Effect of Model Capacity Constraints on Belief State Representations
Studying Generalization and Abstraction
Computational mechanics provides a formal framework for understanding the concepts needed to perform optimal prediction. Abstraction and generalization seem core to the function of intelligent systems, but are not yet …