Skip to content
Sprint projectNov 15, 2026

False Assurance

Mohit Goyal, Ananya Bhattacharya · Team Two Bits

Submitted to AI x Epistemics Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Community Notes has AI note writers, and AI agents read the notes. The platform's quality gates check whether a note looks sourced (links load, raters tick "cites high-quality sources", readers see "Rated helpful"), not whether the cited page supports the claim. A source-check tool could tell readers, but its verdict can itself be wrong. We gave three chat LLMs (Claude Opus 5.5, Claude Haiku 5.5, Gemini 3.8 Flash) 80 Helpful notes whose source support two LLM judges agreed on and human labels confirmed for 78, with right or wrong verdicts, with and without the checked passage (1,920 answers). With a verdict only, the readers inherited the checker's errors: the verdict moved belief by 35–62.5 percentage points and real support by −5 to +7.5, and accuracy fell to 21–30% when the verdict was wrong. The Claude readers mostly withdrew belief as "unsure", not "no". Adding the checked passage cut dependence on the verdict by 26–51 points in all three models, and a wrong verdict shown with its passage left readers no less accurate than no tool. The protection was one-sided: readers rejected false alarms but rarely false assurances, and under a false assurance an unrelated passage worked as well as the real one. A page-level check of 200 paired notes found that 52–56% (human labels) to 68% (LLM judge) of readable cited pages do not fully support the claim, with no detectable AI–human difference. Verification signals for AI readers should carry their evidence and treat "supported" as the risky verdict.

Reviews

Judging this Sprint?

Review “False Assurance”

Your public critique appears on the project page without your name. Reviews of strong projects are shared with funders, as you choose below.

Not shown on the project page.

Shown on the project page, without your name.

Only the Apart team reads this, and funders if you agree below.

Decline to score

Only when the quality of the writing makes the work impossible to follow. Say why in your private critique; the project page shows only that a judge declined, not why.

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.
No infohazards
Sharing your name *

When we share a strong project with funders, your name, scores and public critique go with it, confidentially; this choice is only about whether your name is public. If you think this project deserves funding, please also endorse it: that carries the most weight.

Share my private critique with funders *

See our Privacy Policy.

  1. This project attends to a timely question that affects our information ecosystems and human levels of trust by evaluating how AI’s “trust” what other AI say. The finding that AI readers mostly just believe an AI source-check tool has implications for the kinds of information humans eventually see and use to inform opinions or act on—even when the information is ungrounded in citations.

    While there are methodological limits to the study (and the authors disclose them), this is a solid submission that should be studied further.

    My largest difficulty was the presentation with 2 studies presented in the same short paper—the abstract beginning with the second one and the paper opening with the first. It was hard to follow some of the conditions labeled in the tables. In this sense the structure and writing clarity took away from the findings.

Cite this project

@misc{goyal2026false,
  title = {{False Assurance}},
  author = {Mohit Goyal and Ananya Bhattacharya},
  year = {2026},
  month = nov,
  note = {Submitted to AI x Epistemics Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/false-assurance-5kn9n4xzcv}},
  url = {https://apartresearch.com/sprints/projects/false-assurance-5kn9n4xzcv}
}
Browse all projects