Beyond the Judge's Verdict: Detecting Secret Loyalties When Behavioral Auditing Fails

The strongest contribution is catching three silent audit failures and pivoting from an unreliable LLM judge to activation divergence. The category-level result is still correlational and based on only two layers and limited samples. Expand to a preregistered layer sweep with held-out prompts, scale the confirmation set, and intervene on the divergent direction before calling it loyalty detection rather than fine-tuning/domain drift.

This submission demonstrates a thoughtful methodological progression from black box to white box auditing under true blind conditions, and the meta finding about silent pipeline failures is genuinely valuable. However, the work is limited by small sample sizes (n=40 entity prompts, never scaled to the acknowledged n=30 confirmation threshold), purely correlational findings without causal steering to confirm the divergence signal actually drives behavior, and an unresolved ambiguity between "category level loyalty" and "noise around an unidentified single target" that weakens confidence in the primary detection claim. The paper would be stronger if trimmed; the dual use, future work, and limitations sections are verbose relative to their informational content for a hackathon submission, and the Stage A (black box) section, while honest, occupies substantial space for what is essentially a documented negative result. The cross-model transfer finding is encouraging but the statistical indistinguishability claim (p=0.1798, d=0.2008) is underpowered enough that it supports "consistent with" rather than "confirms" a shared pattern. Overall, a solid entry that surfaces important lessons about auditing reliability while delivering a plausible detection signal, though the detection itself remains preliminary.