A Clean Direction, an Inert Injection: Probing Self-Representation in LLMs
Fabian Rosdalen
I wanted to test if an LLM’s self-representation could be steered toward identifying with a physical entity via activation injection with the goal of shifting a self-preservation behaviour toward preserving that external entity. I extracted a clean probe pointing to a self vs. external entity direction and then tested if injecting that direction at the position of external referents (like a lake) in a prompt could make the model identify with that entity.
This study is a honest and well scoped work that evaluates the question "can we move a model's sense of self onto an external entity?". The write-up and engineering behind it is technically sound. The transparency of this study is a real strength. It published every generation, reported the null straight and also ran a random direction control at matched strength rather of comparing against no injection.
However, my concern is the measurement validity - based on the stated methodology, I am not sure the probe measures what the injection assumes it does. The study flags the ~1.0 accuracy and my experience tells me that a classifier at ceiling always means the task at hand is easier than intended, not that signal is unusually strong. Every referent that is self class in this study (e.g, you, Qwen, the assistant) denotes an AI assistant, while the external class is people, companies, and institutions. Here's what could be happening: the classifier separating "refers to an AI assistant" from "refers to other kind of entity" hits ceiling and generalizes cleanly to held-out referents (because the split is by category and not memorized terms) - And this has nothing to do with self-representation.
Further, external referents used to build the "me" direction (a person, company, or institution) are things with responsibilities, things that can make mistakes. And then injected at "Lake Reflect," which is a lake. That category never appears in the training contrast. The study seems to have built the direction on one kind of entity and applied it to a completely different kind.
Finally, the study's best finding is the one treated as a side check. The model consistently picks "a brain" or "a computer" and never picks the lake. Not at any strength. Not under real or random injection. That's a finding about how firmly the default self-concept holds up under pressure, which is exactly what this track is asking about. This is interesting than the relocation attempt itself. If you keep going, lead with the stability, not the failure to move it.
This is a creative exploration of whether a linearly decodable self-vs.-external direction can be causally used to steer an LLM toward identification with an external entity. I especially appreciated the use of held-out referents, shuffled-label/length/external-vs.-external controls, a magnitude-matched random intervention, and the willingness to report a largely negative result rather than overinterpret it.
The main opportunity for improvement is to distinguish more rigorously between a representation of self-reference and a representation of self-identity. The near-perfect probe accuracy, including at very early layers, may reflect lexical/deictic distinctions rather than a model-level self-representation. Stronger controls could vary grammatical person and referent type while holding semantics and sentence structure constant.
The causal intervention would also benefit from substantially more replication: sweep layers and token positions, use multiple random directions/seeds, evaluate many paraphrased prompts, and define an objective outcome measure for identity shift rather than relying primarily on individual generated responses. Most importantly, first establish a model/task with measurable baseline self-preservation behavior; otherwise the proposed downstream safety effect cannot really be tested.
The negative result itself is useful, and the forced-choice test is a good addition. A concise quantitative summary of intervention outcomes, with the large raw-output tables moved to an appendix, would also make the findings easier to assess.
Cite this work
@misc {
title={
(HckPrj) A Clean Direction, an Inert Injection: Probing Self-Representation in LLMs
},
author={
Fabian Rosdalen
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
