Jun 3, 2024
Belief State Representations in Transformer Models on Nonergodic Data
Junfeng Feng, Wanjie Zhong, Doroteya Stoyanova, Lennart Finke
Summary
We extend research that finds representations of belief spaces in the activations of small transformer models, by discovering that the phenomenon also occurs when the training data stems from Hidden Markov Models whose hidden states do not communicate at all. Our results suggest that Bayesian updating and internal belief state representation also occur when they are not necessary to perform well in the prediction task, providing tentative evidence that large transformers keep a representation of their external world as well.
Cite this work:
@misc {
title={
Belief State Representations in Transformer Models on Nonergodic Data
},
author={
Junfeng Feng, Wanjie Zhong, Doroteya Stoyanova, Lennart Finke
},
date={
6/3/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
