Skip to content
Apart Research
Apart Research
Contact us
Sprint projectAug 16, 2026New Delhi

Stress-Testing LLM Preference Elicitation: When Consistency Does Not Imply Validity

Shriyam Baloni · Team Kappa_Null

Submitted to Digital Minds Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Stress-Testing LLM Preference Elicitation: When Consistency Does Not Imply Validity

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

We investigate whether stable LLM choices provide reliable evidence of preference like states. We develop a multi-method diagnostic framework spanning stated choice, consequential commitment and compensatory consequential choice, while testing presentation, semantic-prior and sampling sensitivity. Across three stimulus domains on one open weight model , we find that apparent stability can arise from surface artifacts or strong directional priors. Notably, raw cross method agreement reached 81.25% , yet chance corrected agreement was negative (Cohen's κ = −0.079) . A compensatory choice diagnostic further produced strong presentation sensitivity rather than a monotonic response. An exploratory temperature extension showed 98.8% trial level agreement across T=0.0 and T=0.7. The central methodological finding is that consistency within an elicitation method is not sufficient evidence of preference validity.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. I think the split into two failure modes works well. Domain 2 is probably the clearest example in the paper: you get 95.0% order invariance, but the model still picks “evict oldest” 97.5% of the time. That makes the point nicely that a preference can be very stable and still be driven by the wrong thing.

    The Cohen’s kappa result is also useful. 81.25% raw agreement sounds fairly high, while κ = -0.079 tells a very different story. I would be a little careful with the negative kappa itself, though. The marginals are extremely concentrated here, which is exactly where kappa starts behaving strangely. I’d probably report the expected chance agreement as well, or maybe a prevalence-adjusted measure, rather than putting too much weight on the fact that kappa is negative.

    M3 was the part I found most interesting conceptually. The compensatory setup is more ambitious than just asking the model which option it prefers, and the flat ρ = 0.010 is a useful null result. I’d keep that.

    The treatment of the 16 failed transmissions also seemed careful, especially since every failure happened on Option A. Doing the sensitivity analysis and then explicitly pointing out that the successful-only subset is itself conditioned on choice is the right way to handle that.

    The biggest limitation for me is still the model. With a quantized 3B checkpoint, strong position effects and default/canonical choices are not especially surprising, and it is hard to know how much of the measured effect would survive on stronger models.

    A simple summary table showing which domains are used for M1, M2, and M3 would also help a lot. I had to keep going back to the methods section to remember which comparison was doing what.

    Read full reviewShow less

Cite this project

@misc{baloni2026stresstesting,
  title = {{Stress-Testing LLM Preference Elicitation: When Consistency Does Not Imply Validity}},
  author = {Shriyam Baloni},
  year = {2026},
  month = aug,
  note = {Submitted to Digital Minds Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/stresstesting-llm-preference-elicitation-when-consistency-does-not-imply-validity-1byd}},
  url = {https://apartresearch.com/sprints/projects/stresstesting-llm-preference-elicitation-when-consistency-does-not-imply-validity-1byd}
}

More from Digital Minds Research Sprint

All 233 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026