Coherent Values, or the Frame That Asked? From Preference Transitivity to Identifiability
Alex Kwon
Reframing a question reverses up to 52% of a model's strongly-held pairwise preferences, and 29% under near-deterministic decoding, while transitivity, the coherence criterion behind claims of emergent value systems, registers nothing. Five models, a frozen 45-pair battery, eight frames, no LLM judge. We propose an identifiability gate separating coverage from coherence, validated on four agents whose mechanism we set: up to 40% of a preference graph is not attributable to the model at all.
The rigor here really stands out! Preregistering before the first run, no LLM judges anywhere, and openly listing which claims you did not show.
Thoughts :
1. Only 2 of 5 models clear the noise floor, and "up to 52%" seems stronger than warranted.
2. 3 repeats is the binding constraint, your own validation shows sensitivity jumps from 0.67 to 1.00 at 5 repeats, so more repeats would help a lot?
3. Would love to see a second domain dataset beyond charity choices.
This project asks whether successful self-prediction by an LLM reflects privileged self-access, or whether an external observer can make the same inference from ordinary features of the model’s outputs. This is a useful distinction: above-chance self-prediction is not, by itself, evidence of introspection if the relevant information is cheaply recoverable from text.
The authors test this in two ways. First, they use a persona-prediction task with a crossed design across different open weight models, aiming to separate self-specific prediction from general classifier capability. Second, they test direct self-prediction, asking models which of two responses they themselves would produce. Hermes-3 succeeds above chance, but a simple “pick the longer response” rule performs even better on the same pairs.
The authors therefore conclude, appropriately, that self-prediction is demonstrated for Hermes-3, but privileged self-access is not.
Strengths
- Strong research question: directly targets a key methodological issue in behavioural introspection research.
- Good execution: distinguishes self-prediction from privileged self-access rather than conflating the two.
- Useful external baselines: the length-only heuristic is especially compelling because it is cheap, matched item-for-item, and outperforms Hermes.
- Good handling of response bias: the authors correctly identify near-constant A/B responding as an elicitation failure rather than a meaningful null.
- Excellent transparency: failed manipulations, post-hoc analyses, sample-size deviations, and changes from preregistration are all clearly disclosed.
- Useful methodological contribution: the surface-leakage and response-bias checks could be valuable beyond this specific experiment.
Limitations
- The style-equalization manipulation is confounded: it removes not only stylistic differences but also much of the underlying behavioural difference between personas, so the collapse to chance cannot establish that style caused the original effect.
- The capability-controlled interaction is not fully identifying: it removes additive competence differences but not predictor-by-target-distribution effects.
- The strongest comparator is post hoc: the length heuristic is compelling, but it should be replicated on fresh data with the rule fixed in advance.
- Limited scope currently, but reasonable for a weekend project: one model lineage, one hidden-property construction, and prompting-only evidence.
Overall assessment: I would rate this as a very strong sprint submission. Its main contribution is methodological: it shows how behavioural self-prediction can look more introspective than it is unless cheap external cues, response biases, and manipulation validity are explicitly tested.
Some suggestions for follow up work:
- Replicate the Hermes result on fresh data with the length rule and other cheap baselines preregistered.
- Develop a style-control manipulation that preserves the behavioural distinction being predicted.
- Test whether self-report adds predictive information beyond a strong external observer, rather than only comparing raw accuracies.
Cite this work
@misc {
title={
(HckPrj) Coherent Values, or the Frame That Asked? From Preference Transitivity to Identifiability
},
author={
Alex Kwon
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
