Adversarial Improvement of Preference Probes

Well the experiment/research area is good, but the 3 out of 4 models are from Qwen, so the no mood difference in the models reflects the family of Qwen. It doesn't generalize to LLMs, for which more data tested across more models of different family would be useful!

This project makes a framing point worth more than the sum of its experiments: measurement under optimization pressure behaves differently from measurement at rest. Using a cheap probe to adversarially hunt for items where it diverges from expensive ground truth, then retraining on those items, is a sensible and reasonably novel transplant of SURF into interpretability tooling — and the motivating example in the discussion (probing immediately for behaviors that would otherwise only surface after 10,000 tokens of generation) is the clearest statement of why this could matter. The scope achieved in a sprint is substantial, and the code and lens/vector artifacts are released.

The strongest empirical result is the question-phrasing finding on Qwen2.5-7B, because it is the one claim you triangulated properly: showing the inflation lives in the stated letter-logit channel but not in the internal probe or revealed choice, and that the gap inverts on other models, is exactly the right decomposition and is a real warning for anyone reading preferences off pairwise logits.

The central claim about probe generalization is where the evaluation design does not yet carry the weight put on it. Two gaps drive most of my concern. First, you note yourself that the held-out items come from the same SURF family with similarly weighted attributes — so the probe is being scored on a distribution the loop is shaping. An independent evaluation set (a different item generator, a different category taxonomy, or human-written items) would make this claim stand up. Second, and more important, there is no matched-compute control: a probe retrained on the same number of randomly generated additional items. Without it, "adversarial selection helped" is not separable from "more data helped," and that control is cheap relative to what you already ran. Relatedly, "significantly improved" in the abstract is not backed by a test statistic, and with one run per model, no seeds, and no confidence intervals, the Llama dip below frozen v0 at cycle 2 means the trend could be noise. Multiple probe seeds and bootstrap CIs would settle this.

There is also a circularity worth confronting directly: your probe's training target is derived from stated pairwise logits — the exact channel your own question-phrasing result shows is contaminated. "The probe mis-judged these items until we added them to training" and "the probe was taught to reproduce a prompt-format artifact" predict the same curve. Training against revealed choice instead, or re-running the generalization result with artifact-affected items excluded, would distinguish them.

The preference-to-emotion null is stated far more strongly than the evidence supports. "A resounding no" across "several different setups" arrives with no n, no effect sizes, no intervals, and no power analysis, yet appears in the abstract and the contributions list. A null across a handful of unreported configurations is not evidence of absence; either report the setups and effect sizes with CIs, or soften this to "we found no evidence of an effect in the configurations we tried."

The distributional claims are descriptive and partly confounded. Bimodality is argued from KDE shape plus Shapiro p-values, but at n≈4,000 Shapiro rejects normality for trivial deviations and says nothing about modality — use a dip test or compare Gaussian-mixture fits by BIC. The cross-family shape comparison is also confounded by per-model anchoring and z-scoring, since differences in utility scale or calibration can manufacture apparent shape differences. And the Qwen2.5-0.5B "GATE FAIL" is dropped from the size sweep without discussion of what failed or what it implies for the sweep's conclusion.

On reproducibility and clarity, the highest-value fixes are concrete. The sentence defining your Thurstonian loss function ends mid-clause and never states the loss. Probe layer is given as "middle or ¾" with no rationale or ablation; probe type and regularization, the number of rephrasings, the SURF attribute pool and reweighting rule, steps per cycle, and the two-tier judge thresholds are all unspecified. Figures have no numbers or captions and are referred to as "this figure," so claims cannot be mapped to evidence; in-plot text is too small to read at page size; and there is no table of headline numbers, leaving readers to eyeball values off plots. Most usefully, split the evaluation protocol into its own subsection, separate from the training protocol — right now the reader has to reconstruct what was held out from where, which is precisely the thing the central claim depends on.

Finally, the disclosure about discovering concurrent work in the final hours is the right call and reflects well on you. But having disclosed it, say what your delta is. The adversarial-loop contribution does look distinct from persona-conditioned probing, and one or two sentences positioning it would serve you better than leaving the comparison open.