Off-trigger-passive-detection

The proposal sharply frames a useful gating question for passive detection of secret model loyalties and lays out thoughtful controls (especially the matched benign fine-tunes) to isolate loyalty-specific signals from generic drift. As pure design work with zero pilot runs or empirical results from the weekend, however, the claims stay entirely speculative and the execution never leaves the planning stage. Running even the described minimum-viable nine-model black-box comparison on one small base would immediately ground the hypotheses and turn the document into something others could build on.

The proposal has a thoughtful experimental design, especially the use of benign matched fine-tunes and seed twins to distinguish loyalty from ordinary training variation. However, the minimum experiment includes relatively few models, so a classifier could learn characteristics of individual fine-tuning runs rather than a general loyalty signal. Using several independent seeds for every condition, testing across more than one base model, and reporting confidence intervals would make the results more reliable.

The paper could also define “neutral” or “off-trigger” prompts more precisely. Some questions in the values interview may indirectly activate deference-related behavior, which would weaken the claim that detection is completely passive. A fixed prompt-selection procedure and human checks for trigger relevance would help. Finally, since this is currently a proposal, even a small pilot experiment would add considerable value by showing whether the suggested features produce a measurable signal above normal model variation.