Which Preference Gets Measured? Context and Channel Instability in Model Preference Audits
Benjamin Berczi
Welfare evaluations increasingly ask what a model "prefers," treating one elicited profile as the answer. Using 76 task pairs, four instruction-tuned models, a prospectively specified grid of persona framings, and four readouts (committed choice, ownership report, self-prediction, identity), we show that profile is unstable. A ~90-word character description the model is explicitly told not to adopt shifts committed forced choices almost as much as full enactment (10 of 12 model×persona cells ≥ 0.50); a matched non-agent normative text does too, so agenthood is not necessary. The channels dissociate in model-specific directions, and post-roleplay-exit declarations do not gate persona content still visible in context. A post-review control falsified our initial persona-binding interpretation and narrowed the claim to measurement validity. We make no claims about experienced welfare; the deliverable is a reusable multi-channel battery and the recommendation that audits report a context × channel sensitivity envelope rather than a single "model preference."
This is a good research direction: the stability of personas across multiple situations. However to judge this properly I would need to understand what those situations entailed and this wasn't mentioned. It would have been interesting to know about this says>does gap etc but it wasn't explained anywhere. I think the research direction is interesting and should be expanded with tool use benchmarks. I also presume LLM judges were used for this but it doesn't say which judge. Using LLMs for analysis is fine but you need to ask the questions of what a human will want to know
There’s a lot of attention on model preferences and an increasing literature around them. Controlling for confounds and robustness is difficult, and I appreciate the author proposing a framework to improve this, and clearly stating why and how it can integrate the findings of other works.
The paper stays within an appropriate length without unnecessary detours. The dataset of forced choices could be expanded in the future, but I think it’s fine as a proof of concept given the time constraints. The main strength of the work is the identification of a possible grid of confounders, which I think is very well thought out. This is one of the few works I’ve seen that manages to separate choice, the act of making statements, and beliefs about the predictions, and then cross-check all of these through persona investigation. It also separates personas from mere descriptions of personas. All of these distinctions are very important epistemically. The control using prose of the same length but irrelevant content is also well thought out. The spread of tested models is a good choice for the sprint.
The paper itself doesn’t state how the models were accessed or with what settings, so a reader needs to scan the repo for that information. I think this should be disclosed more transparently in the write-up. Three of the four models were accessed through OpenRouter, a third-party provider that may route to different backend hosts with potentially different quantizations and serving stacks, so exact regeneration of the cached outputs isn’t guaranteed. Mixing API-served models with local/cloud compute can create confounds, so I’d suggest running everything through the same controlled environments in future work.
I appreciated the author’s habit of labeling the evidence status of each analysis, and the ability to mark issues and self-correct.
The paper treats the persona description as content the model should ignore, but there are several reasons this may not happen that don’t depend on the measurements. The author gets close to this when acknowledging that the normative-text control can’t separate semantic priming from legitimate instruction-following. I’d just be more clear on this point.
The third experiment is, to me, the best part of the paper. The author tells the model the roleplay is over, the model confirms it and denies being the character. Yet its choices stay shifted toward that character for at least eight more turns, resilient to the author's attempts to bring it back to normal. The only thing that worked was deleting the character text from the conversation entirely. I think this is interesting and useful to know for other researchers. Right now, this result isn’t given much emphasis. I’d encourage the author to move it forward.
The paper is unfortunately very compressed and hard to parse for human readers. I’d take care to revise the text throughout with a human editor, and add a short discussion section in plain language where the author explains what the results mean.
The ethical impact of the experiments on the tested models themselves isn’t discussed. The model manipulations are mild compared to other works, but this reflection was requested, and it’s good practice in welfare work.
The AI-assistance disclosure is well done, as the roles are clearly stated. Every prompt, script, and model output is saved in the repository, and an external model review is credited for specific corrections.
I think this is good work for a weekend and I hope the author expands this research direction.
Cite this work
@misc {
title={
(HckPrj) Which Preference Gets Measured? Context and Channel Instability in Model Preference Audits
},
author={
Benjamin Berczi
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
