Skip to content
Apart Research
Apart Research
Contact us
Sprint projectMar 23, 2026Walnut Creek

Resample With Trusted Comparison

Chris Harig

Submitted to AI Control Hackathon 2026. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Resample With Trusted Comparison

Open in a new tab (opens in new tab)
Share

We validate a technique that compares resamples of suspicious actions to actions from trusted models for safety and incrimination, and an automated red team attack-transcript pipeline.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. The idea sounds interesting but it's not clear to me how this is much better than just normal defer to resampling since I would assume most of the time max(trusted) < min(untrusted) if untrusted is flagging the threshold

    This paper would have been more interesting if there was a safety curve and safety / usefulness curves to make it easier to visualize. Also, a larger sampler size would have been better. Also using sonnet to edit the qwen transcripts makes the results more confusing - using a more clear methodology would have helped.

  2. This is solid exploration of an interesting Blue team technique. The red teaming pipeline is also interesting. There wasn't as much validation of the protocol and exploration of it as I'd like though this may not have been doable due to time/resource contraints. I think that rewriting transcripts with sonnet was probably worse than just using an attack policy with sonnet directly.

    It seems like the main way it seems better than DTR is reducing false positive rate. I'd be interested in ROC curves for both DTR and RTC so we can have more clear comparisions of them and their efficacy as well.

    something claude flagged that i agree with:

    - the different denominators in Table 1 (11/16 for DTR vs 12/15 for RTC) are unexplained and a bit suspicious

Cite this project

@misc{harig2026resample,
  title = {{Resample With Trusted Comparison}},
  author = {Chris Harig},
  year = {2026},
  month = mar,
  note = {Submitted to AI Control Hackathon 2026, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/resample-with-trusted-comparison-u6bu}},
  url = {https://apartresearch.com/sprints/projects/resample-with-trusted-comparison-u6bu}
}

More from AI Control Hackathon 2026

All 125 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026