Skip to content
Apart Research
Apart Research
Contact us
Sprint projectAug 16, 2026Bangalore

Where Did That Confidence Come From?

Rohit Kale , Pranjal Panghal · Team Source Unknown

Submitted to Digital Minds Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Where Did That Confidence Come From?

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

One way to make claims about model metacognition more testable is to give a model a task whose answer requires causally controlled information from its own hidden computation and cannot be recovered from the visible transcript. We do this by perturbing a hidden confidence state in Gemma 3 while keeping the visible question and answer unchanged, then asking the model to report the hidden intervention through a counterbalanced code. The confidence direction has far greater causal efficacy at the answer-adjacent token than one token later and produces the strongest delayed readout among 40 matched directions. Yet an intervention isolated to Answer A’s cache row also influences a later confidence report about Answer B. The project therefore demonstrates functional access to hidden confidence information while showing that detection alone does not establish where that information originated.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. - Authors do KV editing and show that: edit confidence trace "attached to A → later confidence query about A changes" but also that "edit confidence trace attached to A → later confidence query about B changes". This is interesting!

    - The empirical tests are well done and have necessary controls (blocking attention as mentioned would be nice!).

    - Understanding how confidence is represented, used, and bound to the particular content it is confidence about is interesting and important.

  2. This is an interesting study with a convincing methodology. On reading the presentation, this reader got lost in the details and could at many places not really follow. One thing that did not become fully clear is how your study differs from [7]:

    “Kumaran et al. [7] provide a task-native state for this test. They showed that Gemma 3 27B forms a representation of answer quality immediately after answering, stores it in the key/value cache, and later retrieves it when reporting verbal confidence. We build on that causally validated state rather than injecting an unrelated concept.”

    To highlight the novelty of your study, you write:

    “Our study differs in two ways. The target is a naturally formed, independently characterized confidence state rather than an arbitrary semantic concept, and the primary specificity comparison uses 39 directions built by relabeling the same source activations. The row-isolation audit then tests an assumption that intervention-report paradigms usually leave implicit: whether the causal trace remains attributable to the state where it was introduced.”

    If I understand correctly, [7] show *that* interfering with the representation of answer quality affects verbal confidence reports, whereas you show that confidence reports are not just affected by this, but are directly based on the representation of answer quality? If not, what exactly did you show? Either way, I recommend clarifying this part of the presentation.

    The methods and results sections use many abbreviations and technical details that I cannot follow. If everyone working on this kind of research understands this, that’s fine, but readers who are not directly involved in this kind of research (like myself) are unlikely to understand this part.

    The discussion and remarks about limitations seem reasonable and plausible.

    Read full reviewShow less

Cite this project

@misc{kale2026where,
  title = {{Where Did That Confidence Come From?}},
  author = {Rohit Kale and Pranjal Panghal},
  year = {2026},
  month = aug,
  note = {Submitted to Digital Minds Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/where-did-that-confidence-come-from-8qe1}},
  url = {https://apartresearch.com/sprints/projects/where-did-that-confidence-come-from-8qe1}
}

More from Digital Minds Research Sprint

All 233 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026