Preferences Under Pressure
Javier Prieto
Recent work finds that language-model choices can be summarized by coherent utility functions. We test whether those inferred preferences survive a realistic change in elicitation. Three GPT models chose between every pair of 27 executable AI tasks under one abstract stated-preference prompt and two consequentially framed prompts, in which limited time or API credits meant that only the selected task could be completed. Each of the 351 task pairs was sampled five times in both option orders, yielding 10,530 choices per model. Bradley-Terry rankings from the two consequential framings agreed closely, but stated rankings
transferred poorly: depending on model and resource framing, Pearson correlations were only .46-.69, and stated utilities predicted consequential choices 12-31 percentage points less accurately than utilities fitted within the target condition. Newer models were more individually consistent under consequential framing, not more invariant: mean Laplace-smoothed pairwise preference strength rose from .70-.72 for GPT-4o-mini to .83-.85 for GPT-5.6 Luna and Terra. These results suggest that utility models can recover stable, condition-specific orderings without identifying a single context-independent preference ranking.
This project runs a well-controlled, full factorial test of a live and increasingly important question: do LLM "preferences" measured under an abstract, no-stakes framing predict the same model's choices when a plausible resource constraint is attached to the decision? Using Mazeika et al.'s exact task battery and validating that a simpler Bradley-Terry model reproduces the original Thurstonian fits almost exactly (r=.981) before switching to it, the design is methodologically careful and the choice to simplify is justified rather than assumed.
The central finding is well-supported: two independently-worded consequential framings (running out of API tokens vs. running out of review time) recover nearly identical rankings from each other (r=.90–.91) while both diverge substantially from the abstract stated-preference ranking (r=.46–.69). That the two consequential framings agree with each other so strongly is important — it suggests the stated-consequential gap reflects something structural about how the model treats "this choice has a real effect" framing, not noise from either individual prompt's wording. The paper's most distinctive result is that this gap does not shrink with capability: newer models are more internally consistent under consequential framing, but the accuracy loss from using stated preferences to predict consequential choices actually grows (not shrinks) for more capable models — a sharper and less intuitive finding than the general "elicitation protocols matter" result already established in adjacent recent work.
Two limitations are worth flagging. First, all three tested models are GPT-family — the paper is honest about this, but it means the capability-widens-the-gap finding is not yet known to generalize across model families, only across a capability gradient within one family. Second, Claude Haiku's high indeterminate rate (83.1%/52.6%) is treated as an informative failure and excluded rather than forced into the analysis, which is the right call methodologically but does mean the study's core quantitative claims rest on a narrower model set than the headline "LLM task preferences" framing might suggest.
This work sits alongside several very recent papers independently stress-testing whether Mazeika-style coherent utilities are behaviorally meaningful, via different mechanisms (elicitation-protocol sensitivity, downstream incentive transfer, advice-vs-performance prediction). The paper cites the elicitation-protocol and downstream-behavior lines but not the closest incentive-transfer paper testing the same underlying paradigm; adding that comparison would sharpen the positioning of what specifically is new here relative to a fast-moving adjacent literature.
This hints at an interesting result that's worth following up on, but some issues weren't able to be addressed by the hackathon deadline, like divergent response formats between arms that were meant to be directly compared (single-token-formatted response versus open-ended). Addressing those issues, expanding the analysis to other model families, and trying multiple framings to reduce the potential overly-specific impact of each one, could be a really interesting research direction.
Cite this work
@misc {
title={
(HckPrj) Preferences Under Pressure
},
author={
Javier Prieto
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


