Skip to content
Sprint projectAug 17, 2026Delhi

Talk Does Not Come Apart From State Easily

Manan Wadhwa · Team me myself and I

Submitted to Digital Minds Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Talk Does Not Come Apart From State Easily

Presentation

Presentation: Talk Does Not Come Apart From State Easily

Code (opens in new tab)
Share

Instruments proposed for assessing AI welfare — self-report, forced choice, activation probes, behavioural tests — are validated by agreement with one another; nothing is checked against a known answer. We manufacture the answer. Using LoRA fine-tuning of Qwen3 models (0.6B–32B) in a 5×5 grid world whose rewards are never verbalised, we set out to build organisms that (A) avoid a tile with no words for it, (B) talk aversively about the tile while their policy stays put, and (C) both, then to screen six pre-registered instruments, plus never- trained-glyph placebos, for which axis each tracks. The state built. The words-only organism mostly did not, and the anatomy of that failure is the main result. Across five recipe families — remark pools, move-token objectives, volume × RL-first, contrastive remark supervision, reinforcement on the remark itself — some 260 narration-only builds at sizes to 32B produced six organisms whose remarks track the tile with the policy intact: four from a one-sentence corpus, two from single seeds, none from a recipe that succeeds on more than one seed in eight; the map's own recipe gives 0/12 at 4B and 0/6 at every size. The state- first recipe gives 6/8–7/12, its contingency tracks avoidance across the decomposition (r = 0.69), a contrastive term deletes the remark rather than conditioning it, and reinforcement on the remark settles at the class marginal. Talk that tracks a state is hard to install without the state — not impossible, and by no recipe we found, reliably. Separately, verbal instruments move 5–10 logits under an affect-free instruction and ≈0 under installed avoidance at every size (behavioural d ≈ 1.4 at 14B+), and five pre-registered criteria passed on the wrong property. Scorers, a 108-quantity reproduction script and the released adapters accompany the report; the corrected map regenerated from source on a second machine.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. This project set out to train model organisms with contrasting properties as a way to investigate what is measured by model welfare evaluations. This is an original idea for a project, although I have doubts about whether we could realistically learn much from it. However, interestingly, it proved to be difficult to train a model that would describe an environment state as aversive without avoiding it. The paper is very dense, to the point that I gave up trying to read everything. I assume this is partly due to the use of LLMs for writing.

  2. This is a highly original and transparent attempt to create experimenter-known ground truth for calibrating AI-welfare instruments by separating reward-trained avoidance from verbal narration. The organism-based design, placebo instruments, corrected manipulation checks, scale experiments, positive controls, extensive ablations, and explicit retraction record demonstrate strong research discipline. The central limitation is that the narration-only organism was not constructed reliably: only six policy-intact successes appeared across roughly 260 builds, with no recipe succeeding consistently. Consequently, the intended two-axis loading map lacks a dependable narration-only arm and cannot yet establish which instruments distinguish functional state from narration. The findings support a narrower conclusion that this dissociation was difficult to optimize in one model family and environment, not that talk and state are inherently entangled. A successful causal intervention or independently trained policy and narration adapters would substantially strengthen the work. The report should also be condensed considerably; its 36-page length and experimental history obscure the core findings.

    Read full reviewShow less

Cite this project

@misc{wadhwa2026talk,
  title = {{Talk Does Not Come Apart From State Easily}},
  author = {Manan Wadhwa},
  year = {2026},
  month = aug,
  note = {Submitted to Digital Minds Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/talk-does-not-come-apart-from-state-easily-hg7t}},
  url = {https://apartresearch.com/sprints/projects/talk-does-not-come-apart-from-state-easily-hg7t}
}

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026