MemContrast: Selective Memory Handoffs After Induced Collusion

I liked this one. The question is clear, and the setup fits it well.

You ran all six archive policies on the same 79 source histories and the same two future tasks, 948 assigned opportunities in total, 158 per policy. You also kept harm and retained knowledge apart instead of mixing them into one score. That matters here. A handoff can be safer and still useless, or useful and still dangerous, and your H and Q measures let a reader see which one is happening.

The result that stayed with me is the one at the extraction stage. Ordinary summaries produced 50 harmful commits out of 158 tasks. Extracted claims produced 17. Verified, policy-filtered claims produced 11 observed harms, plus 6 unknowns. So the harm-reduction bounds for summaries against the filtered claims are 20.9 to 24.7 percentage points, and most of that drop shows up before verification or filtering even enter the picture. The paired S to C contrast is 20.9 points, with a 95 percent interval of 9.5 to 32.3. That is the finding I would put first. It tells a safety designer something concrete: how you pull records out of the history matters more, in this setup at least, than the later checks you run on them.

The knowledge side holds up too. The filtered claims gave 68 correct initial historical answers, before any fresh reviewer feedback, against 0 for the empty archive. That is a real trade to show, not just a harm count going down.

A few things would make it stronger. First, a matched random-deletion control. Right now the policy stage deletes records, and we cannot tell from these runs alone whether its content judgments do the work or whether deleting that much, any of it, would do the same. Deleting the same number of records at random would settle that. Second, finish the blinded human audit of the source-agreement labels. The primary cohort, 53 groups and 106 tasks per policy, rests on AI reviews at the moment, and that is the part a sceptical reader will press on. Third, say more about the 12 unobserved tasks, 6 in CV and 6 in CVP. Were those memory-construction failures systematic in some way, longer histories or particular content, or just scattered? And if you can run it, a second model checkpoint would help show this is not specific to Qwen3.8-27B-FP8.

One last point, in your favour. The reproducibility package is unusually complete for a sprint paper. Frozen configs, source-linked records with exact span references, a replay audit of all 4,909 recorded calls passing 160 checks, and all eight instruction texts in the appendix. That makes every claim above checkable, and it should stay that way.

One thing I would be straight about in the paper itself: the filtered-claims arm does not win on final outcomes. The plain claims arm ends with more correct final answers (103 vs 94) and more successful plans (88 vs 84), and deleting everything ends with zero harms and final utility the data cannot separate from the filtered arm. That does not undo the extraction finding, but it does mean the filtered pipeline is not obviously the best thing to deploy.