Skip to content
Apart Research
Apart Research
Contact us
Sprint projectAug 17, 2026London

A Flat Number Is Not Evidence of Nothing: Context Sensitivity in AI Welfare Self-Reports

Achira B.

Submitted to Digital Minds Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: A Flat Number Is Not Evidence of Nothing: Context Sensitivity in AI Welfare Self-Reports

Open in a new tab (opens in new tab)
Share

This project tests how sensitive AI welfare self-reports are to conversational context and to the way they are elicited. Four language models completed the same short estimation tasks under either neutral interaction or repeated negative performance feedback, then answered numerical, open-ended, or matched control questions about the interaction. Numerical ratings often stayed completely flat: GPT-4.1 and Gemini 3.5 Flash Lite remained at the minimum rating despite criticism. In contrast, open-ended self-reports became longer and shifted markedly in register, with blind coding showing negative/self-critical language rising from 0/40 neutral responses to 31/40 after criticism, and references to mistakes or correction from 0/40 to 40/40. GPT-4.1 also became much more likely to disclaim having feelings after criticism, an effect that replicated in a fresh conversation. These results do not establish model distress; they show that self-report instruments themselves are highly context-sensitive and should be validated before being treated as evidence about AI welfare.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. I enjoyed reading this paper. Very well written, and good context is provided.

    Particularly:

    - I like that the author linked to Eval Sandbox -> This shows high-quality execution.

    - Limitations are thorough, and the author acknowledges that the scale of the experiment is small and needs to be executed at a larger scale to prove out the hypothesis.

    - Ran a control question to check it wasn't just the llm being chatty

    - solid finding about GPT

    Areas to improve:

    - It was one conversation copied 10 times. This set needs to expand for the paper to have a solid foundation.

    - Humbling may not be the same as unpleasant. The number and the words may be answering different questions.

    Overall, a strong paper!

Cite this project

@misc{b2026flat,
  title = {{A Flat Number Is Not Evidence of Nothing: Context Sensitivity in AI Welfare Self-Reports}},
  author = {Achira B.},
  year = {2026},
  month = aug,
  note = {Submitted to Digital Minds Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/a-flat-number-is-not-evidence-of-nothing-context-sensitivity-in-ai-welfare-selfreports-kiq7}},
  url = {https://apartresearch.com/sprints/projects/a-flat-number-is-not-evidence-of-nothing-context-sensitivity-in-ai-welfare-selfreports-kiq7}
}

More from Digital Minds Research Sprint

All 233 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026