Language Models Underestimate How Recent Work Will Change Their Choices

This work examines how models' choices are affected by tasks they have recently performed, together with their ability to predict these effects. There are some curious results that might turn out to be revealing on further examination, such as the finding that GPT-5.6 Luna's forecasts were less accurate when it was in the same context in which the choice would be made. The paper would benefit from more explanation of the motivation for the study.

This is decently clear and reported. The author prospectively records predictions, retains failed ones, counterbalances labels and order, tests multiple phrasings, and openly discusses the mismatch between the forecast and behavioral conditions. The full-transcript versus one-line-summary reversal is the most genuinely interesting finding.

My concern is that the underlying behavior is not especially consequential or surprising. Models continuing the task they have just repeatedly performed can arise from ordinary discourse continuation, lexical recency, inferred user intent, or local consistency. A model-generated percentage failing to anticipate the exact magnitude of that effect is not yet strong evidence about self-knowledge, particularly because the forecast describes imposed work while the main behavioral condition presents ordinary user requests.

The author’s experimental skill would be better applied to whether models can forecast safety-relevant changes in their own behavior: refusal erosion, deception, goal drift, susceptibility to fine-tuning, memory-induced commitments, or behavioral changes caused by long-horizon tool use.