A Linear Answerability Signal That the Model Does Not Act On
Mackenzie O'Brien · Team Mac Labs AI
Submitted to AI x Epistemics Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
We ask whether Phi-3.5-mini-instruct (3.8B parameters, 8-bit) carries a linearly readable signal for whether a question is answerable, and whether its stated confidence or abstention behavior reflects that signal. On SQuAD 2.0 (1,200 balanced questions, passage-disjoint 60/20/20 splits, test n=240), a logistic probe on the last prompt token reaches an area under the ROC curve (AUROC) of 0.906 (95% confidence interval, CI, 0.867–0.948) at layer 18, chosen on validation data. Verbalized confidence reaches 0.621, token-level and sampling signals 0.665–0.688, question length 0.569, and probes trained on shuffled labels 0.492. The behavioral counterpart is weak: the model abstains on 5% of unanswerable test items, and an explicit UNANSWERABLE option raises this to 15% with precision 1.0 (18 of 18). On 46.7% of all test items the probe says unanswerable while the model commits with confidence at least 0.8, and those items are wrong 82.1% of the time, against 52.5% overall. SelfAware gives a probe AUROC of 0.993, but a source-only feature reaches 1.000, so we report it as a confound; transfer between the two datasets is weak (0.58 and 0.67). We claim a linear signal that predicts answerability within SQuAD under length and shuffle controls, from one model, one run and one prompt. We do not claim that the model knows it does not know.
Reviews
No public critique yet.
Cite this project
@misc{obrien2026linear,
title = {{A Linear Answerability Signal That the Model Does Not Act On}},
author = {Mackenzie O'Brien},
year = {2026},
month = nov,
note = {Submitted to AI x Epistemics Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/a-linear-answerability-signal-that-the-model-does-not-act-on-feuho1zptz}},
url = {https://apartresearch.com/sprints/projects/a-linear-answerability-signal-that-the-model-does-not-act-on-feuho1zptz}
}Browse all projects