Beaten by a Cheap Surface Classifier: A Capability-Controlled Test of Privileged Self-Access

• I like the central point that successful self-prediction is weaker evidence if a very cheap external predictor can do even better.
• In the end there is not so much evidence backing the claim that models do not have privileged self-access. It's a kind of warning against overinterpreting self-prediction.
• A limitation for me is that this is a relatively constrained classification setting: the models are given already-generated text and asked to identify a property of it, or to choose which of two existing replies they would produce. This seems easier and importantly different from predicting in advance whether a model will, for example, report that it prefers X.
• Sentences like “The originally preregistered estimand is positive on the leakiest set, and we do not bury it,” “This is our largest limitation, and we found it ourselves, after the fact,” and “confirmation is not preregistration” (and many others) that are probably LLM-generated make the document hard and frustrating to read.

The research question is whether a model’s ability to predict its own behavior reflects privileged self-access, or whether an external observer using the same textual evidence can do just as well or better. The authors test this using capability-matched model observers, cheap surface-feature baselines, and controls intended to separate genuine self-prediction from stylistic cues.

This is a strong and careful project. The main contribution is the methodology - it shows how above chance self-prediction can be confounded by cheap surface information, and introduces useful checks for surface leakage and response bias. I also appreciated the transparency around failed manipulations, post hoc analyses, and reduced sample size.

The main limitation is that the style-equalization intervention also weakened the underlying behavioral distinction between persona conditions, so this feels like a confound: the resulting collapse toward chance cannot cleanly establish that style caused the original effect. The crossed capability control also does not fully rule out predictor-by-target differences.

I would therefore interpret the result conservatively: privileged self-access is not demonstrated, and cheap external cues are an important confound. The most valuable next step would be a preregistered replication on fresh data with the length rule and other cheap comparators fixed in advance, alongside a better manipulation that suppresses surface cues while preserving the behavioral distinction.

Overall, this is methodologically thoughtful, transparent, and a promising foundation for further work.

This project asks whether successful self-prediction by an LLM reflects privileged self-access, or whether an external observer can make the same inference from ordinary features of the model’s outputs. This is a useful distinction: above-chance self-prediction is not, by itself, evidence of introspection if the relevant information is cheaply recoverable from text.

The authors test this in two ways. First, they use a persona-prediction task with a crossed design across different open weight models, aiming to separate self-specific prediction from general classifier capability. Second, they test direct self-prediction, asking models which of two responses they themselves would produce. Hermes-3 succeeds above chance, but a simple “pick the longer response” rule performs even better on the same pairs.

The authors therefore conclude, appropriately, that self-prediction is demonstrated for Hermes-3, but privileged self-access is not.

Strengths

• Strong research question: directly targets a key methodological issue in behavioural introspection research.
• Good execution: distinguishes self-prediction from privileged self-access rather than conflating the two.
• Useful external baselines: the length-only heuristic is especially compelling because it is cheap, matched item-for-item, and outperforms Hermes.
• Good handling of response bias: the authors correctly identify near-constant A/B responding as an elicitation failure rather than a meaningful null.
• Excellent transparency: failed manipulations, post-hoc analyses, sample-size deviations, and changes from preregistration are all clearly disclosed.
• Useful methodological contribution: the surface-leakage and response-bias checks could be valuable beyond this specific experiment.

Limitations

• The style-equalization manipulation is confounded: it removes not only stylistic differences but also much of the underlying behavioural difference between personas, so the collapse to chance cannot establish that style caused the original effect.
• The capability-controlled interaction is not fully identifying: it removes additive competence differences but not predictor-by-target-distribution effects.
• The strongest comparator is post hoc: the length heuristic is compelling, but it should be replicated on fresh data with the rule fixed in advance.
• Limited scope currently, but reasonable for a weekend project: one model lineage, one hidden-property construction, and prompting-only evidence.

Overall assessment: I would rate this as a very strong sprint submission. Its main contribution is methodological: it shows how behavioural self-prediction can look more introspective than it is unless cheap external cues, response biases, and manipulation validity are explicitly tested.

Some suggestions for follow up work:

• Replicate the Hermes result on fresh data with the length rule and other cheap baselines preregistered.
• Develop a style-control manipulation that preserves the behavioural distinction being predicted.
• Test whether self-report adds predictive information beyond a strong external observer, rather than only comparing raw accuracies.