The Dynamic Self: Probing AI Survival Instincts Across Shifting Identities and Unseen Domains
Ankur Pandey
We isolated the exact internal signal an AI uses to protect itself, proving it relies on a general concept of survival rather than simple word-association. By assigning the model different identities, we watched its internal geometry align to protect its newly assigned "self," confirming it dynamically tracks its own boundaries. This exact same "harm-to-me" instinct then successfully flagged threats in hundreds of unseen network logs, proving the self-preservation drive is an abstract, highly transferable concept.
This study asks a clear and intriguing question: “Is there a direction in the residual stream that tracks harm and benefit to the model itself, rather than the same events happening to something else?” The methodology is careful and seeks to exclude two main confounds (tracking sentiment and tracking entity names).
To highlight the novelty of your study, you write: “We add two things: LEACE erasure of the sentiment confound (Belrose et al., 2023), and a measured reliability ceiling.” I did not really understand the “measured reliability ceiling”.
The methods section focuses on how the study controlled for confounds. This is very dense and provides very short descriptions of each confound and control. I found it hard to follow. I would recommend a more detailed explanation.
The results section is quite technical and hard to follow for readers like me (who are not directly involved this kind of research)
The discussion and remarks about limitations seem reasonable and plausible. In particular, it is noted that a relatively stable direction in activation space that tracks self-harm and -benefit is found, but that no intervention was performed, so it could be that the model does not actually use this information. This suggests one of several relevant issues for future research.
The core result is a good one: the self-welfare direction follows the in-context designation rather than the entity name, and the sign-flip against the ungrounded control is a convincing way to show it. The confound discipline is the strongest part, the LEACE sentiment erasure, the measured reliability ceiling instead of a fixed cosine threshold, and the willingness to discard their own pre-planned threshold when a simulation showed a null model would clear it. My reservations are about what the evidence licenses. The work is correlational, one model at one layer, with no intervention along the direction, so nothing yet connects this representation to behavior. The designation is handed to the model in a prompt line, which is not evidence of a standing self-model, and roughly a fifth of the response is still generic valence after erasure. Cross-domain agreement is also modest at 60 to 70 percent of ceiling, so a substantially shared direction is the honest claim, and the paper says so. Steering the direction and testing whether self-preservation behavior moves with it is the obvious next step and would change what this result means.
Decent rigor for a sprint. Counterfactual quadruplets, the ungrounded baseline, the pre-registered encoding simulation, and above all the measured reliability ceiling show excellent experimental judgment.
The question itself carried a high prior. A capable instruct model resolving an explicit "you are Node-A" and binding valence to the referent is expected behavior, and models are already known to bind entities to attributes internally via binding ID vectors (Feng & Steinhardt 2023). The clean sign flip confirms competence we could have predicted; the informative outcome would have been failure.
The deeper gap is that nothing here is specific to self. Run the same pipeline with "the auditor's server is Node-A" and probe harm to that server. Matching geometry means you found designated-referent binding and the pronoun did all the work; privileged geometry for "you" would be the actual discovery, and it costs about a dollar. I'd run that before steering. Then connect the direction to behavior, since a self-model earns the name when the system uses it for control. Check whether probe activation predicts choices in shutdown or sacrifice dilemmas even before intervening. Your future-work list is well aimed; for steering, prefer a difference-in-means direction, since those generalize as well as other probing techniques while being more causally implicated in outputs (Marks & Tegmark 2023), and for inferred identity, SAD's task formats are a ready template (Laine et al. 2024). You've built exactly the pipeline these experiments need. Run them.
Cite this work
@misc {
title={
(HckPrj) The Dynamic Self: Probing AI Survival Instincts Across Shifting Identities and Unseen Domains
},
author={
Ankur Pandey
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
