Does Persona Sensitivity Predict Self-Report Reliability? An Exploratory Cross-Model Study of Persona Perturbations and Elicited Reasoning
Jack Lakkapragada
Digital-minds research often relies on models’ self-reports about preferences, identity, and potentially welfare-relevant states. We began with the hypothesis that these reports might share a common failure mode: models whose answers change more when the assistant persona is perturbed might also show greater divergence between elicited reasoning and final responses. We tested this across Claude Sonnet, Llama 3.3 70B, and Qwen 2.5 72B using five preference- and identity-related queries and three prompting conditions. After correcting the analysis to use query-level estimates, we found no stable cross-model relationship. TF-IDF Spearman correlations were +0.60, −0.20, and −0.70 respectively; under semantic similarity they became +0.10, −0.30, and +0.60. Qwen’s sign reversal was especially informative. Our results suggest that persona sensitivity and scratchpad–output consistency should be treated as distinct behavioral diagnostics rather than interchangeable evidence about a model’s underlying preferences or introspective reliability.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Does Persona Sensitivity Predict Self-Report Reliability? An Exploratory Cross-Model Study of Persona Perturbations and Elicited Reasoning
},
author={
Jack Lakkapragada
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


