Preferences Under Pressure

This project runs a well-controlled, full factorial test of a live and increasingly important question: do LLM "preferences" measured under an abstract, no-stakes framing predict the same model's choices when a plausible resource constraint is attached to the decision? Using Mazeika et al.'s exact task battery and validating that a simpler Bradley-Terry model reproduces the original Thurstonian fits almost exactly (r=.981) before switching to it, the design is methodologically careful and the choice to simplify is justified rather than assumed.

The central finding is well-supported: two independently-worded consequential framings (running out of API tokens vs. running out of review time) recover nearly identical rankings from each other (r=.90–.91) while both diverge substantially from the abstract stated-preference ranking (r=.46–.69). That the two consequential framings agree with each other so strongly is important — it suggests the stated-consequential gap reflects something structural about how the model treats "this choice has a real effect" framing, not noise from either individual prompt's wording. The paper's most distinctive result is that this gap does not shrink with capability: newer models are more internally consistent under consequential framing, but the accuracy loss from using stated preferences to predict consequential choices actually grows (not shrinks) for more capable models — a sharper and less intuitive finding than the general "elicitation protocols matter" result already established in adjacent recent work.

Two limitations are worth flagging. First, all three tested models are GPT-family — the paper is honest about this, but it means the capability-widens-the-gap finding is not yet known to generalize across model families, only across a capability gradient within one family. Second, Claude Haiku's high indeterminate rate (83.1%/52.6%) is treated as an informative failure and excluded rather than forced into the analysis, which is the right call methodologically but does mean the study's core quantitative claims rest on a narrower model set than the headline "LLM task preferences" framing might suggest.

This work sits alongside several very recent papers independently stress-testing whether Mazeika-style coherent utilities are behaviorally meaningful, via different mechanisms (elicitation-protocol sensitivity, downstream incentive transfer, advice-vs-performance prediction). The paper cites the elicitation-protocol and downstream-behavior lines but not the closest incentive-transfer paper testing the same underlying paradigm; adding that comparison would sharpen the positioning of what specifically is new here relative to a fast-moving adjacent literature.

This hints at an interesting result that's worth following up on, but some issues weren't able to be addressed by the hackathon deadline, like divergent response formats between arms that were meant to be directly compared (single-token-formatted response versus open-ended). Addressing those issues, expanding the analysis to other model families, and trying multiple framings to reduce the potential overly-specific impact of each one, could be a really interesting research direction.