Beaten by a Cheap Surface Classifier: A Capability-Controlled Test of Privileged Self-Access
Jaswin Chinthala, Ubayd Hattas
We test whether language models possess privileged self-access by evaluating behavioural self-prediction against an equal-or-lower-cost external observer. Using a capability-controlled crossed design (9,269 trials), we find that while Hermes-3 can predict its own outputs (0.719 balanced accuracy), a simple 1-feature length rule (0.808) and a 21-feature surface classifier (0.831) perform better. We conclude that self-prediction alone does not demonstrate privileged access, and we release a surface-leakage gate for future introspection research.
- I like the central point that successful self-prediction is weaker evidence if a very cheap external predictor can do even better.
- In the end there is not so much evidence backing the claim that models do not have privileged self-access. It's a kind of warning against overinterpreting self-prediction.
- A limitation for me is that this is a relatively constrained classification setting: the models are given already-generated text and asked to identify a property of it, or to choose which of two existing replies they would produce. This seems easier and importantly different from predicting in advance whether a model will, for example, report that it prefers X.
- Sentences like “The originally preregistered estimand is positive on the leakiest set, and we do not bury it,” “This is our largest limitation, and we found it ourselves, after the fact,” and “confirmation is not preregistration” (and many others) that are probably LLM-generated make the document hard and frustrating to read.
The research question is whether a model’s ability to predict its own behavior reflects privileged self-access, or whether an external observer using the same textual evidence can do just as well or better. The authors test this using capability-matched model observers, cheap surface-feature baselines, and controls intended to separate genuine self-prediction from stylistic cues.
This is a strong and careful project. The main contribution is the methodology - it shows how above chance self-prediction can be confounded by cheap surface information, and introduces useful checks for surface leakage and response bias. I also appreciated the transparency around failed manipulations, post hoc analyses, and reduced sample size.
The main limitation is that the style-equalization intervention also weakened the underlying behavioral distinction between persona conditions, so this feels like a confound: the resulting collapse toward chance cannot cleanly establish that style caused the original effect. The crossed capability control also does not fully rule out predictor-by-target differences.
I would therefore interpret the result conservatively: privileged self-access is not demonstrated, and cheap external cues are an important confound. The most valuable next step would be a preregistered replication on fresh data with the length rule and other cheap comparators fixed in advance, alongside a better manipulation that suppresses surface cues while preserving the behavioral distinction.
Overall, this is methodologically thoughtful, transparent, and a promising foundation for further work.
Cite this work
@misc {
title={
(HckPrj) Beaten by a Cheap Surface Classifier: A Capability-Controlled Test of Privileged Self-Access
},
author={
Jaswin Chinthala, Ubayd Hattas
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
