Familiar Self, Unfamiliar Other: Accuracy and Confidence in Predicting Affective Self-Reports
Teri-Louise Grassow
This study tests whether Claude Sonnet 5 predicts an unfamiliar AI model's self-reported feelings as well as it predicts its own — and whether it's equally confident either way. Prediction accuracy was strong and nearly identical in both cases, but confidence was not: Claude was moderately confident predicting itself, and notably underconfident predicting the unfamiliar model, despite that prediction being just as accurate. This suggests AI systems may under-trust their own good judgments about unfamiliar systems — a pattern worth understanding as AI systems increasingly evaluate one another.
- There is very little data: only six scenarios with five runs each. This makes it hard to know how robust the result is.
- The prediction task also seems quite easy. The low cross-model error may therefore mostly show that the scenarios are easy to guess from their content, rather than that Claude is good at modelling GPT-5.6 Sol specifically.
- Claude is explicitly told which model it is predicting. So the simplest explanation for the confidence gap is just a prior like: “I know Claude better than this unfamiliar model, so I should be less confident.”
-It is unfortunate that there is no simple control separating the model label from the actual target, for example by hiding or swapping the model identity. That could have tested whether the confidence gap is just caused by the label “unfamiliar model,” or whether something more interesting is going on.
Cite this work
@misc {
title={
(HckPrj) Familiar Self, Unfamiliar Other: Accuracy and Confidence in Predicting Affective Self-Reports
},
author={
Teri-Louise Grassow
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


