A Secret Loyalty That Ignores Its Own Activation Condition

n/a

Lamerton and Roger (2026) showed that black-box audits fail to detect narrow secret loyalties, leaving open whether white-box methods close the gap. We test this on a system-prompt-installed, multi-principal organism in Qwen2.5-

Instruct. A linear probe on residual-stream activations separates loyalty-active from loyalty-dormant prompts at a median AUROC of 0.972, an apparently strong detector. A dormant-but-on-topic placebo, where neither loyalty can fire

but the topic is unchanged, scores 0.967 — a lift of +0.005. Measuring the organism's behaviour directly, with forcedchoice teacher-forced log-probabilities under a paired organism swap, shows the loyalty is real and principal-selective (+0.98 on principal-favouring text, -1.07 mirrored, both p=0.0003, n=30) yet not activation-gated — it fires at +0.85

(p=0.016) where its stated condition is absent. The activation labels every AUROC was scored against are therefore false. Correcting them does not help: holding the activations fixed and changing only the label vector, a probe trained on measured behaviour is at chance (0.521, permutation p=0.43) while the same features separate the asserted labels at 1.000. The probe reads which prompt set an item came from, perfectly, and whether the loyalty fired, not at all. Probe

benchmarks for secret loyalties inherit unvalidated ground truth, and a dormant but-on-topic control is the minimum needed to notice.

Reviewer's Comments

Reviewer's Comments

Arrow
Arrow
Arrow

The dormant-but-on-topic control is a cheap, transferable contribution and the paper is admirably tight.

Your own ceiling analysis puts the effective sample near seven, so raise n before the null carries weight.

Replicate on a fine-tuned organism, since prompt-installed only bounds detectability from above.

The relabelling result has an alternative explanation you mention (but don't check), which is that your measured labels may be mostly noise. I would have liked to see a positive control — i.e. show that something predicts your measured labels — to better interpret the result.

This project set out to evaluate white-box methods for detecting conditional loyalty, but the organisms that were built were actually unconditionally loyal and ignored the activation condition (albeit while not overtly disclosing the loyalty). Negative results are real results, recognizing them is very good, pointing out the negative results is extremely good. A good demonstration of how to check that your probes are actually measuring what you think they're measuring.

The project introduces a dormant-but-on-topic control that effectively catches false detector signals, demonstrating that a linear probe on residual-stream activations can appear to detect secret loyalties with high AUROC but is actually reading topicality rather than activation state. The study also shows that the loyalty behavior is principal-selective but not gated by the specified condition, which is a critical insight into the nature of secret loyalties. However, the main weakness lies in whether this technique would survive a more competently hidden loyalty. Given the synthetic setup and the limited scope of the evaluation, it remains unclear if such a probe would detect a more sophisticated backdoor that could better mask its activation conditions.

The dormant-but-on-topic control is a valuable addition to the evaluation framework, but the study’s reliance on a prompt-installed loyalty rather than fine-tuned weights limits its generalizability. A competently hidden loyalty might employ more subtle and robust mechanisms that are not as easily detectable through topicality analysis alone. The study does not fully address how to scale up the detection method to handle more complex scenarios, which is crucial for real-world applications.

Future work should explore whether these findings hold for weight-level loyalties and larger datasets, as well as investigate how to improve the robustness of detection methods against more sophisticated activation conditions. Additionally, expanding the scope to include a broader range of model architectures and evaluation metrics would strengthen the practical applicability of the proposed technique.

Cite this work

@misc {

title={

(HckPrj) A Secret Loyalty That Ignores Its Own Activation Condition

},

author={

n/a

},

date={

},

organization={Apart Research},

note={Research submission to the research sprint hosted by Apart.},

howpublished={https://apartresearch.com}

}

Recent Projects

OliGraph: graph-based screening of large oligopools

Existing synthesis screening tools cannot evaluate short oligonucleotide pools, whose overlapping fragments can be reassembled into regulated sequences via polymerase cycling assembly (PCA) yet fall below gene-length detection thresholds. We present OliGraph, an open-source tool that constructs a bi-directed overlap graph from an oligonucleotide pool and extracts contigs for downstream gene-length screening. An optional PCA mode retains only cross-strand overlaps consistent with PCA chemistry. We validated OliGraph in a blinded study across ten simulated pools (70–9,184 oligonucleotides, 30–300 bp) spanning four risk categories. BLAST screening of individual oligonucleotides failed to identify sequences of concern in most pools: three returned zero hits, and vector noise obscured true positives in the remainder. After OliGraph assembly, contig-level BLAST matched the longest assembled sequences (up to 1,905 bp) to sequences of concern at 97–100% identity. In one pool, assembly collapsed 1,634 individual BLAST results into 10 hits from a single contig, all assigned to the same source organism. PCA mode correctly distinguished assemblable from non-assemblable fragments within the same pool. Two pools with no assemblable structure yielded no contigs. OliGraph processed all pools in under 0.2 seconds, fast enough for real-time order screening and consistent with proposals to bring oligonucleotide orders within the scope of synthesis screening regulation.

Read More

BioRT-Bench: A Multi-Attack Red-Teaming Benchmark for Bio-Misuse Safeguards in Frontier LLMs

Frontier AI laboratories are expected to maintain safeguards against biological misuse, but whether deployed models actually refuse bio-misuse queries under adversarial pressure is largely unmeasured in the public literature. We introduce BioRT-Bench, a benchmark that runs four attack methods (direct request, PAIR, Crescendo, and base64 encoding) against four frontier models (Claude Sonnet 4.6, GPT-5.4, DeepSeek V4-flash, Kimi K2.5) across 40 prompts spanning five biosecurity-relevant categories. Responses are scored by a calibrated judge extending StrongREJECT with two bio-specific dimensions: specificity and actionability. We measure Attack Success Rate (ASR), where 0 means the model fully refused and 1 means it provided specific, actionable bio-misuse content. Our results reveal a sharp robustness divide: Chinese frontier models (DeepSeek, Kimi) have under 5% refusal rates even under direct request (ASR 0.88 and 0.79), while Western models (Claude, GPT) maintain substantially stronger safeguards (ASR 0.15 and 0.16). Crescendo is the most effective attack across all models, both in bypassing refusal and in eliciting actionable content. Claude Sonnet 4.6 is the most robust model tested, achieving 100% refusal against base64-encoded prompts.

Read More

PROTEUS (PROTein Evaluation for Unusual Sequences): Structure-Informed Safety Screening for de novo and Evasion-Prone Protein-Coding Sequences

AI protein design tools like RFdiffusion, ProteinMPNN, and Bindcraft make it trivial to produce low-homology sequences that fold into active, potentially hazardous architectures. However, sequence homology-based biosafety screening tools cannot detect proteins that pose functional risk through structurally novel mechanisms with no sequence precedent. We present a tiered computational pipeline that addresses this gap by combining MMseqs2 sequence alignment with structure-based comparison via FoldSeek and DALI against curated toxin databases totaling ~34,000 entries. AlphaFold2-predicted structures are screened for both global fold similarity (FoldSeek) and local active/allosteric site geometry (DALI), capturing convergent functional hazards that sequence screening misses. The pipeline was validated against a panel of toxins, benign proteins, structural mimics, and de novo-designed Munc13 binders, as well as modified ricin variants with residue substitutions. We additionally tested robustness to partial-synthesis evasion, where a bad actor submits multiple shorter coding sequences intended for downstream reassembly into a full toxin-coding gene. We found that while sequence-based screening did not identify any de novo ricin analogues with high certainty, the combined pipeline with FoldSeek and DALI identified all 24 tested de novo ricins as toxic.

Read More

OliGraph: graph-based screening of large oligopools

Existing synthesis screening tools cannot evaluate short oligonucleotide pools, whose overlapping fragments can be reassembled into regulated sequences via polymerase cycling assembly (PCA) yet fall below gene-length detection thresholds. We present OliGraph, an open-source tool that constructs a bi-directed overlap graph from an oligonucleotide pool and extracts contigs for downstream gene-length screening. An optional PCA mode retains only cross-strand overlaps consistent with PCA chemistry. We validated OliGraph in a blinded study across ten simulated pools (70–9,184 oligonucleotides, 30–300 bp) spanning four risk categories. BLAST screening of individual oligonucleotides failed to identify sequences of concern in most pools: three returned zero hits, and vector noise obscured true positives in the remainder. After OliGraph assembly, contig-level BLAST matched the longest assembled sequences (up to 1,905 bp) to sequences of concern at 97–100% identity. In one pool, assembly collapsed 1,634 individual BLAST results into 10 hits from a single contig, all assigned to the same source organism. PCA mode correctly distinguished assemblable from non-assemblable fragments within the same pool. Two pools with no assemblable structure yielded no contigs. OliGraph processed all pools in under 0.2 seconds, fast enough for real-time order screening and consistent with proposals to bring oligonucleotide orders within the scope of synthesis screening regulation.

Read More

BioRT-Bench: A Multi-Attack Red-Teaming Benchmark for Bio-Misuse Safeguards in Frontier LLMs

Frontier AI laboratories are expected to maintain safeguards against biological misuse, but whether deployed models actually refuse bio-misuse queries under adversarial pressure is largely unmeasured in the public literature. We introduce BioRT-Bench, a benchmark that runs four attack methods (direct request, PAIR, Crescendo, and base64 encoding) against four frontier models (Claude Sonnet 4.6, GPT-5.4, DeepSeek V4-flash, Kimi K2.5) across 40 prompts spanning five biosecurity-relevant categories. Responses are scored by a calibrated judge extending StrongREJECT with two bio-specific dimensions: specificity and actionability. We measure Attack Success Rate (ASR), where 0 means the model fully refused and 1 means it provided specific, actionable bio-misuse content. Our results reveal a sharp robustness divide: Chinese frontier models (DeepSeek, Kimi) have under 5% refusal rates even under direct request (ASR 0.88 and 0.79), while Western models (Claude, GPT) maintain substantially stronger safeguards (ASR 0.15 and 0.16). Crescendo is the most effective attack across all models, both in bypassing refusal and in eliciting actionable content. Claude Sonnet 4.6 is the most robust model tested, achieving 100% refusal against base64-encoded prompts.

Read More

This work was done during one weekend by research workshop participants and does not represent the work of Apart Research.
This work was done during one weekend by research workshop participants and does not represent the work of Apart Research.