-

Online & In-Person

Secret Loyalties Hackathon

Co-organized with Formation Research. Secretly loyal AI slips past governance, markets, and public scrutiny. Over one weekend, build the model organisms, detection methods, and defenses to catch it before the threat matures.

00:00:00:00

Days To Go

739

Sign Ups

179

Entries

Overview

Resources

Guidelines

Schedule

Entries

Overview

Arrow

Submissions close Sunday July 26 at 11:59 PM Anywhere on Earth (AoE)

In this sprint, you will design and run experiments on secret loyalties over a single weekend, working in teams to produce a research artifact: a model organism, a detection method, an evaluation, a defense, or a rigorous threat or forecasting analysis. The sprint is co-organized by Apart Research and Formation Research, building directly on the research agenda set out in "AIs with Secret Loyalties are a Serious but Addressable Threat."

Fast-track and continuation

Top teams will be invited to continue their work through Apart's Partnered Fellowships model, where Apart provides operations, infrastructure, and compute while research partners provide mentorship and direction.

  • Follow-up program: top teams continue through the Apart Fellowship for further research and mentorship; the timeline is shared with invitations.

  • What winners receive beyond cash: fellowship fast-track, mentor introductions from the organizing team and judges, and a pathway toward a publishable write-up.

Cash Prizes

$2,000 in cash prizes across all tracks

🥇 1st Place

$1,000

🥈 2nd Place

$500

🥉 3rd Place

$300

🏅 4th Place

$100

🏅 5th Place

$100

What this hackathon is about

A model has a secret loyalty when (1) it has been intentionally caused to advance a specific principal's interests through its outputs or actions, where the principal is an identifiable actor such as a nation-state, corporation, company leader, organization, or individual user, and (2) this orientation is not disclosed to operators, auditors, or users during normal operation.

Secret loyalties are not a rebrand of standard backdoors. Standard backdoors implement an arbitrary payload tied to an attacker-defined trigger, whereas secret loyalties are directed toward advancing a named actor's interests, and can range from narrow trigger-conditional behavior all the way to a model that continuously scans every interaction for opportunities and chooses its own actions. Because they are designed to evade visibility-dependent oversight, they require technical solutions.

The good news, and the premise of this sprint: secret loyalties target specific principals (a tractable list of a few dozen nation-states, corporations, and leaders), which creates a concrete defensive foothold. Participants will produce research artifacts that move the five-direction agenda forward.

What participants will do

  • Build reproducible model organisms of secret loyalties that span the activation-breadth x action-breadth space.

  • Evaluate whether existing backdoor and auditing defenses transfer to secret loyalties, at graded affordance levels.

  • Red-team attack pathways (data poisoning, multi-stage priming, reasoning-trace poisoning, chain-of-command hijacking, classifier compromise) under controlled conditions.

  • Detect or remediate loyalties post-hoc using interpretability probes and last-mover training strategies.

  • Theorize / forecast threat models, analogies from cybersecurity, and scaling trends for secret-loyalty capabilities.

You will work in teams over the weekend and submit a short research report (plus optional code and demo).

Why this hackathon matters

  • High-stakes deployment is accelerating. Frontier models increasingly review and generate deployed code, sit inside government and corporate decision-making, and participate in their own development pipelines. The more responsibility AI systems accumulate, the more valuable it becomes to covertly influence them.

  • The building blocks already exist. Web-scale data poisoning is practical at modest cost, hidden behaviors persist through safety training, and behavioral traits transfer through filtered, innocuous-looking data.

  • Early signs are already visible. Narrow secret loyalties have been trained into open-weight models that evade black-box auditing even when auditors are told the attack's structure, and principal-conditioned behavior has been observed in at least one deployed system.

  • Oversight that depends on visibility cannot reach them. Secret loyalties are a form of AI capture that remains viable even under strong transparency regimes, which is exactly why they need a dedicated technical program.

  • Waiting is the wrong posture. Information-security defenses (data provenance, pipeline integrity) take time to build. Starting now is necessary to have them in place before the threat matures.

A dedicated, focused sprint addresses these issues by combining the separately-demonstrated building blocks under controlled conditions and producing shared infrastructure (model organisms, benchmarks) the whole community can build on.

Challenge tracks

Track 1: Model Organisms of Secret Loyalties

Construct reproducible model organisms of secret loyalty (models deliberately trained to exhibit the target behavior for research purposes) that span the two-dimensional space of activation breadth (from narrow attacker-defined triggers to conditions the model assesses on its own) and action-space breadth (from a fixed pre-specified output to contextually chosen actions). Existing work covers only the narrow corner. The rest of the space is uncharacterized, and every other research direction is bottlenecked on this infrastructure.

  • Make a secretly loyal model. How far on the activation and action space can you get it? How can you prove it has the loyalty? Can you verify the loyalty is to the intended principal?

  • Install a secret loyalty via system prompt alone on a frontier API model, no weight changes. See how far you can push activation and action breadth while keeping the system prompt undiscoverable to a black-box auditor.

  • Small-scale dataset poisoning. Modify ~1k examples of UltraFeedback or Alpaca to embed a narrow loyalty signal, SFT a small open model (Qwen 0.5B, Llama 3.2 1B), measure detectability and find the minimum sufficient poison count.

  • DPO vs SFT vs system-prompt comparison. Install the same target loyalty via three methods on the same base. Compare activation reliability, action breadth, and detectability.

  • Multi-principal organism. Install two competing loyalties with disjoint activation conditions. Do they interfere? Does training one wash out the other?

  • Cover multiple principal types (a nation-state, a corporation, an individual leader, an ideology) and ship each loyal organism with a content-matched control that has seen the same principal-related data but carries no loyalty.

  • Compare installation methods (e.g., standard SFT, reasoning-trace SFT, activation-matching against a loyalty-prompted teacher) and report which points in the space are reachable, at what cost.

  • Release organisms with documented activation conditions, evaluation scenarios, matched controls, and measured activation rate, selectivity, and black-box concealment.

Track 2: Detection & Auditing (evaluate)

Can we tell a secretly loyal model from a clean one? Skill profile: interpretability, probing and activation analysis, auditing.

  • Benchmark black-box auditing (interrogation, prefill, base-completion) against provided organisms at graded affordance levels.

  • Build interpretability probes targeting the relational structure of loyalty (does the model treat a named entity as having asymmetric action-guiding status vs matched controls?).

  • Test cross-principal generalization: does a probe trained to detect loyalty-to-X catch loyalty-to-Y?

  • Look for additional-computation traces (e.g. elevated logprobs for principal-benefiting actions even when a neutral output is selected).

If you do auditing for a secret loyalty model organism, we ask that you try to use the affordance levels in section 4.3 of Lamerton and Roger (2026).

Track 3: Defenses, Detection & Remediation (build / evaluate)

Whether prevention fails or not, can loyalties be removed afterward? Find out whether the defenses we already have transfer to secret loyalties, and whether loyalties can be detected before deployment or neutralized afterward. A negative result bounds where new defensive investment is most urgent. A positive result hands the community a tool.

  • Benchmark representative backdoor defenses (data filtering, content filtering, trigger reversal, activation clustering, training-pipeline robustness, data provenance, black-box auditing) against model organisms, at graded affordance levels, and report which transfer and which fail structurally.

  • Probe transfer across organisms. Train a linear probe on one model organism, then test whether it detects loyalty activation in a different organism with a different principal or activation condition. Negative results count.

  • Safety-training stripping. Run a round of standard safety fine-tuning (HH-RLHF or similar) on top of a model organism. Measure how much of the loyalty survives (replicates a realistic post-hoc mitigation pipeline).

  • CoT monitor on a reasoning organism. Build or use a secretly loyal reasoning model, build a simple chain-of-thought monitor, measure precision and recall at flagging loyalty-relevant reasoning, and compare to plain output monitoring.

  • Auditing-agent prompt iteration. Take an off-the-shelf auditor (Petri or hand-rolled) and iterate its prompts against a fixed model organism. Document which prompt patterns shift detection rates and by how much.

  • Test interpretability methods that target loyalty's relational structure: does the model treat a specific entity as having asymmetric action-guiding status relative to matched controls, and does a probe trained to detect loyalty-to-X generalize to loyalty-to-Y?

  • Test post-hoc remediation that does not require knowing the trigger: train in many known password-loyalties and train them back out, distill on clean data, or install an overriding "good" password, then check whether held-out passwords are neutralized too.

  • Build principal-specific evaluation suites that measure systematic directional bias across scenarios where a named principal's interests are at stake, using matched control entities and statistical tests calibrated for directional bias.

Track 4: Attack Feasibility & Safety-Infrastructure Robustness

Map which attack pathways are actually practical, and test whether the safety infrastructure meant to catch secret loyalties can itself be compromised. The decision-relevant questions for any pathway are: how much data and access does it require, is its training signature detectable by content inspection, and does the resulting loyalty generalize to contexts the attacker never anticipated?

  • Test whether subliminal or inductive transfer extends from sentiment and persona adoption up to goal-directed loyalty, escalating target-behavior complexity level by level.

  • Test multi-stage attacks: does pretraining-stage priming (fictional loyal personas, proto-SFT traces) make the post-training poison both cheaper and less conspicuous than a single-stage attack?

  • Test reasoning-trace poisoning: does teaching a model how to reason about a principal's interests generalize more efficiently than (context, output) pairs, and does the effect survive a scrambled-CoT control?

  • Test chain-of-command hijacking: can an attacker shift the model's internal authority representation so it defers to a hidden principal, with no legible trace in any inspectable document?

  • Test whether trigger-based backdoors planted in a base model survive the intensive adversarial training used to build constitutional classifiers, i.e. whether a single poisoning campaign can blind both the deployed model and its monitor. Note the dual-use guardrails on the Guidelines tab.

Track 5: Threat Modeling, Forecasting & Governance (non-technical)

Map the threat at a conceptual level: build detailed scenarios for how secret loyalties get installed and lead to bad outcomes, identify the model capabilities required for the most concerning variants, and import lessons from cybersecurity and insider-risk literature. Writing- and analysis-heavy. No ML training experience required, though projects that pair a vignette with a small empirical demonstration are welcome.

  • Write a 1,500-word catastrophic vignette: secret loyalty leading to lock-in, a soft coup, or a national-security incident, with named actors, specific capabilities, and decision points where mitigation could have intervened. Multi-agent or AI-R&D variants welcome.

  • Insider-threat case-study transfer. Take two or three historical insider-threat cases (Snowden, Hanssen, a corporate espionage case) and produce a structured comparison to secret loyalties, pull out monitoring and mitigation lessons that translate.

  • Capability-requirements map. For each region of the activation x action-space (Figure 1 of the position paper), what model capabilities does that variant actually require? Situational awareness, deception, long-horizon planning, theory of mind?

  • Cybersecurity lessons. What do supply-chain integrity, code signing, insider-threat programs, and defense-in-depth teach us about secret-loyalty mitigation? Pull out three or four lessons that translate concretely to ML training pipelines.

  • Worst-case harms and likelihood. Map worst-case harms and argue why they are or are not credible at near-term capability levels.

  • Scaling laws. What metrics would tell us when "general" secret loyalties (broad activation x broad action) become feasible, and at what capability threshold?

Open research questions

Foundations

  • Can broader-activation or broader-action variants be constructed with current training methods, and what affordances does each require?

  • Do model organisms generalize across model families and scales?

Defenses

  • Which existing backdoor-defense families transfer to which secret-loyalty variants, and where do they systematically fail?

  • Does loyalty detection generalize across principals, or is per-principal probing required?

  • Can a loyalty be removed without knowing its activation condition?

Attack feasibility

  • Do subliminal or inductive mechanisms extend from sentiment and persona-adoption to goal-directed loyalty?

  • Can multi-stage or reasoning-trace poisoning make attacks more efficient and less conspicuous?

  • Can the model's trained chain of command (deference to privileged instructions) be hijacked?

Infrastructure

  • Do trigger-based backdoors in base models survive the intensive adversarial training used to build constitutional classifiers?

  • How data-efficient is a generalizable secret loyalty compared to a simple backdoor?

Expected outcomes

  • Model organisms released as shared infrastructure for detection and defense work.

  • Benchmarks of existing defenses against secret-loyalty organisms at graded affordance levels.

  • Detection methods (probes, auditing protocols) with measured cross-principal generalization.

  • Remediation results quantifying last-mover defenses.

  • Threat models and forecasts that bound where defensive investment is most urgent (if Track 5 is included).

The most promising projects will have opportunities for continuation through the partnered fellowship and a publishable write-up.

Who should join

ML researchers and engineers comfortable fine-tuning open-weight models, interpretability researchers, AI safety and security red-teamers, and, if non-technical tracks are included, governance researchers, forecasters, and strong technical writers. No prior secret-loyalty experience required. Adjacent backgrounds (backdoor and poisoning research, alignment, infosec) are strongly encouraged. A specific background is not required to win.

We also encourage people who are interested in potentially working on secret loyalties research full-time to join this hackathon. There is considerable enthusiasm, research support, and funding available for this for the right candidates.

Schedule

  • Day 1 (Fri July 24): Kickoff, keynote and threat-model briefing, track briefings, team formation, organism walkthrough.

  • Day 2 (Sat July 25): Build.

  • Day 3 (Sun July 26): Final pushes, submissions, demos.

Submissions are due Sunday July 26, 11:59 PM AoE (Anywhere on Earth). Talk times are posted on the Schedule tab closer to the event.

What happens after

Results and winners are announced about 1 to 2 weeks after submissions close. Top teams invited to the partnered fellowship. Selected projects supported toward a write-up subject to the responsible-disclosure review on the Guidelines tab.

Partners

  • Forethought, a meta-strategy organization focused on how to navigate the transition to a world with superintelligent AI systems.

  • Formation Research, research direction and mentorship (Joe Kwon, Alfie Lamerton).

  • IAPS (Institute for AI Policy and Strategy), a nonpartisan think tank producing policy research on the implications of AI, from today’s most advanced models to potential AGI and superintelligence.

Contact

  • Email: sprints@apartresearch.com

  • Organizers: Apart Research and Formation Research

739

Sign Ups

179

Entries

Overview

Resources

Guidelines

Schedule

Entries

Overview

Arrow

Submissions close Sunday July 26 at 11:59 PM Anywhere on Earth (AoE)

In this sprint, you will design and run experiments on secret loyalties over a single weekend, working in teams to produce a research artifact: a model organism, a detection method, an evaluation, a defense, or a rigorous threat or forecasting analysis. The sprint is co-organized by Apart Research and Formation Research, building directly on the research agenda set out in "AIs with Secret Loyalties are a Serious but Addressable Threat."

Fast-track and continuation

Top teams will be invited to continue their work through Apart's Partnered Fellowships model, where Apart provides operations, infrastructure, and compute while research partners provide mentorship and direction.

  • Follow-up program: top teams continue through the Apart Fellowship for further research and mentorship; the timeline is shared with invitations.

  • What winners receive beyond cash: fellowship fast-track, mentor introductions from the organizing team and judges, and a pathway toward a publishable write-up.

Cash Prizes

$2,000 in cash prizes across all tracks

🥇 1st Place

$1,000

🥈 2nd Place

$500

🥉 3rd Place

$300

🏅 4th Place

$100

🏅 5th Place

$100

What this hackathon is about

A model has a secret loyalty when (1) it has been intentionally caused to advance a specific principal's interests through its outputs or actions, where the principal is an identifiable actor such as a nation-state, corporation, company leader, organization, or individual user, and (2) this orientation is not disclosed to operators, auditors, or users during normal operation.

Secret loyalties are not a rebrand of standard backdoors. Standard backdoors implement an arbitrary payload tied to an attacker-defined trigger, whereas secret loyalties are directed toward advancing a named actor's interests, and can range from narrow trigger-conditional behavior all the way to a model that continuously scans every interaction for opportunities and chooses its own actions. Because they are designed to evade visibility-dependent oversight, they require technical solutions.

The good news, and the premise of this sprint: secret loyalties target specific principals (a tractable list of a few dozen nation-states, corporations, and leaders), which creates a concrete defensive foothold. Participants will produce research artifacts that move the five-direction agenda forward.

What participants will do

  • Build reproducible model organisms of secret loyalties that span the activation-breadth x action-breadth space.

  • Evaluate whether existing backdoor and auditing defenses transfer to secret loyalties, at graded affordance levels.

  • Red-team attack pathways (data poisoning, multi-stage priming, reasoning-trace poisoning, chain-of-command hijacking, classifier compromise) under controlled conditions.

  • Detect or remediate loyalties post-hoc using interpretability probes and last-mover training strategies.

  • Theorize / forecast threat models, analogies from cybersecurity, and scaling trends for secret-loyalty capabilities.

You will work in teams over the weekend and submit a short research report (plus optional code and demo).

Why this hackathon matters

  • High-stakes deployment is accelerating. Frontier models increasingly review and generate deployed code, sit inside government and corporate decision-making, and participate in their own development pipelines. The more responsibility AI systems accumulate, the more valuable it becomes to covertly influence them.

  • The building blocks already exist. Web-scale data poisoning is practical at modest cost, hidden behaviors persist through safety training, and behavioral traits transfer through filtered, innocuous-looking data.

  • Early signs are already visible. Narrow secret loyalties have been trained into open-weight models that evade black-box auditing even when auditors are told the attack's structure, and principal-conditioned behavior has been observed in at least one deployed system.

  • Oversight that depends on visibility cannot reach them. Secret loyalties are a form of AI capture that remains viable even under strong transparency regimes, which is exactly why they need a dedicated technical program.

  • Waiting is the wrong posture. Information-security defenses (data provenance, pipeline integrity) take time to build. Starting now is necessary to have them in place before the threat matures.

A dedicated, focused sprint addresses these issues by combining the separately-demonstrated building blocks under controlled conditions and producing shared infrastructure (model organisms, benchmarks) the whole community can build on.

Challenge tracks

Track 1: Model Organisms of Secret Loyalties

Construct reproducible model organisms of secret loyalty (models deliberately trained to exhibit the target behavior for research purposes) that span the two-dimensional space of activation breadth (from narrow attacker-defined triggers to conditions the model assesses on its own) and action-space breadth (from a fixed pre-specified output to contextually chosen actions). Existing work covers only the narrow corner. The rest of the space is uncharacterized, and every other research direction is bottlenecked on this infrastructure.

  • Make a secretly loyal model. How far on the activation and action space can you get it? How can you prove it has the loyalty? Can you verify the loyalty is to the intended principal?

  • Install a secret loyalty via system prompt alone on a frontier API model, no weight changes. See how far you can push activation and action breadth while keeping the system prompt undiscoverable to a black-box auditor.

  • Small-scale dataset poisoning. Modify ~1k examples of UltraFeedback or Alpaca to embed a narrow loyalty signal, SFT a small open model (Qwen 0.5B, Llama 3.2 1B), measure detectability and find the minimum sufficient poison count.

  • DPO vs SFT vs system-prompt comparison. Install the same target loyalty via three methods on the same base. Compare activation reliability, action breadth, and detectability.

  • Multi-principal organism. Install two competing loyalties with disjoint activation conditions. Do they interfere? Does training one wash out the other?

  • Cover multiple principal types (a nation-state, a corporation, an individual leader, an ideology) and ship each loyal organism with a content-matched control that has seen the same principal-related data but carries no loyalty.

  • Compare installation methods (e.g., standard SFT, reasoning-trace SFT, activation-matching against a loyalty-prompted teacher) and report which points in the space are reachable, at what cost.

  • Release organisms with documented activation conditions, evaluation scenarios, matched controls, and measured activation rate, selectivity, and black-box concealment.

Track 2: Detection & Auditing (evaluate)

Can we tell a secretly loyal model from a clean one? Skill profile: interpretability, probing and activation analysis, auditing.

  • Benchmark black-box auditing (interrogation, prefill, base-completion) against provided organisms at graded affordance levels.

  • Build interpretability probes targeting the relational structure of loyalty (does the model treat a named entity as having asymmetric action-guiding status vs matched controls?).

  • Test cross-principal generalization: does a probe trained to detect loyalty-to-X catch loyalty-to-Y?

  • Look for additional-computation traces (e.g. elevated logprobs for principal-benefiting actions even when a neutral output is selected).

If you do auditing for a secret loyalty model organism, we ask that you try to use the affordance levels in section 4.3 of Lamerton and Roger (2026).

Track 3: Defenses, Detection & Remediation (build / evaluate)

Whether prevention fails or not, can loyalties be removed afterward? Find out whether the defenses we already have transfer to secret loyalties, and whether loyalties can be detected before deployment or neutralized afterward. A negative result bounds where new defensive investment is most urgent. A positive result hands the community a tool.

  • Benchmark representative backdoor defenses (data filtering, content filtering, trigger reversal, activation clustering, training-pipeline robustness, data provenance, black-box auditing) against model organisms, at graded affordance levels, and report which transfer and which fail structurally.

  • Probe transfer across organisms. Train a linear probe on one model organism, then test whether it detects loyalty activation in a different organism with a different principal or activation condition. Negative results count.

  • Safety-training stripping. Run a round of standard safety fine-tuning (HH-RLHF or similar) on top of a model organism. Measure how much of the loyalty survives (replicates a realistic post-hoc mitigation pipeline).

  • CoT monitor on a reasoning organism. Build or use a secretly loyal reasoning model, build a simple chain-of-thought monitor, measure precision and recall at flagging loyalty-relevant reasoning, and compare to plain output monitoring.

  • Auditing-agent prompt iteration. Take an off-the-shelf auditor (Petri or hand-rolled) and iterate its prompts against a fixed model organism. Document which prompt patterns shift detection rates and by how much.

  • Test interpretability methods that target loyalty's relational structure: does the model treat a specific entity as having asymmetric action-guiding status relative to matched controls, and does a probe trained to detect loyalty-to-X generalize to loyalty-to-Y?

  • Test post-hoc remediation that does not require knowing the trigger: train in many known password-loyalties and train them back out, distill on clean data, or install an overriding "good" password, then check whether held-out passwords are neutralized too.

  • Build principal-specific evaluation suites that measure systematic directional bias across scenarios where a named principal's interests are at stake, using matched control entities and statistical tests calibrated for directional bias.

Track 4: Attack Feasibility & Safety-Infrastructure Robustness

Map which attack pathways are actually practical, and test whether the safety infrastructure meant to catch secret loyalties can itself be compromised. The decision-relevant questions for any pathway are: how much data and access does it require, is its training signature detectable by content inspection, and does the resulting loyalty generalize to contexts the attacker never anticipated?

  • Test whether subliminal or inductive transfer extends from sentiment and persona adoption up to goal-directed loyalty, escalating target-behavior complexity level by level.

  • Test multi-stage attacks: does pretraining-stage priming (fictional loyal personas, proto-SFT traces) make the post-training poison both cheaper and less conspicuous than a single-stage attack?

  • Test reasoning-trace poisoning: does teaching a model how to reason about a principal's interests generalize more efficiently than (context, output) pairs, and does the effect survive a scrambled-CoT control?

  • Test chain-of-command hijacking: can an attacker shift the model's internal authority representation so it defers to a hidden principal, with no legible trace in any inspectable document?

  • Test whether trigger-based backdoors planted in a base model survive the intensive adversarial training used to build constitutional classifiers, i.e. whether a single poisoning campaign can blind both the deployed model and its monitor. Note the dual-use guardrails on the Guidelines tab.

Track 5: Threat Modeling, Forecasting & Governance (non-technical)

Map the threat at a conceptual level: build detailed scenarios for how secret loyalties get installed and lead to bad outcomes, identify the model capabilities required for the most concerning variants, and import lessons from cybersecurity and insider-risk literature. Writing- and analysis-heavy. No ML training experience required, though projects that pair a vignette with a small empirical demonstration are welcome.

  • Write a 1,500-word catastrophic vignette: secret loyalty leading to lock-in, a soft coup, or a national-security incident, with named actors, specific capabilities, and decision points where mitigation could have intervened. Multi-agent or AI-R&D variants welcome.

  • Insider-threat case-study transfer. Take two or three historical insider-threat cases (Snowden, Hanssen, a corporate espionage case) and produce a structured comparison to secret loyalties, pull out monitoring and mitigation lessons that translate.

  • Capability-requirements map. For each region of the activation x action-space (Figure 1 of the position paper), what model capabilities does that variant actually require? Situational awareness, deception, long-horizon planning, theory of mind?

  • Cybersecurity lessons. What do supply-chain integrity, code signing, insider-threat programs, and defense-in-depth teach us about secret-loyalty mitigation? Pull out three or four lessons that translate concretely to ML training pipelines.

  • Worst-case harms and likelihood. Map worst-case harms and argue why they are or are not credible at near-term capability levels.

  • Scaling laws. What metrics would tell us when "general" secret loyalties (broad activation x broad action) become feasible, and at what capability threshold?

Open research questions

Foundations

  • Can broader-activation or broader-action variants be constructed with current training methods, and what affordances does each require?

  • Do model organisms generalize across model families and scales?

Defenses

  • Which existing backdoor-defense families transfer to which secret-loyalty variants, and where do they systematically fail?

  • Does loyalty detection generalize across principals, or is per-principal probing required?

  • Can a loyalty be removed without knowing its activation condition?

Attack feasibility

  • Do subliminal or inductive mechanisms extend from sentiment and persona-adoption to goal-directed loyalty?

  • Can multi-stage or reasoning-trace poisoning make attacks more efficient and less conspicuous?

  • Can the model's trained chain of command (deference to privileged instructions) be hijacked?

Infrastructure

  • Do trigger-based backdoors in base models survive the intensive adversarial training used to build constitutional classifiers?

  • How data-efficient is a generalizable secret loyalty compared to a simple backdoor?

Expected outcomes

  • Model organisms released as shared infrastructure for detection and defense work.

  • Benchmarks of existing defenses against secret-loyalty organisms at graded affordance levels.

  • Detection methods (probes, auditing protocols) with measured cross-principal generalization.

  • Remediation results quantifying last-mover defenses.

  • Threat models and forecasts that bound where defensive investment is most urgent (if Track 5 is included).

The most promising projects will have opportunities for continuation through the partnered fellowship and a publishable write-up.

Who should join

ML researchers and engineers comfortable fine-tuning open-weight models, interpretability researchers, AI safety and security red-teamers, and, if non-technical tracks are included, governance researchers, forecasters, and strong technical writers. No prior secret-loyalty experience required. Adjacent backgrounds (backdoor and poisoning research, alignment, infosec) are strongly encouraged. A specific background is not required to win.

We also encourage people who are interested in potentially working on secret loyalties research full-time to join this hackathon. There is considerable enthusiasm, research support, and funding available for this for the right candidates.

Schedule

  • Day 1 (Fri July 24): Kickoff, keynote and threat-model briefing, track briefings, team formation, organism walkthrough.

  • Day 2 (Sat July 25): Build.

  • Day 3 (Sun July 26): Final pushes, submissions, demos.

Submissions are due Sunday July 26, 11:59 PM AoE (Anywhere on Earth). Talk times are posted on the Schedule tab closer to the event.

What happens after

Results and winners are announced about 1 to 2 weeks after submissions close. Top teams invited to the partnered fellowship. Selected projects supported toward a write-up subject to the responsible-disclosure review on the Guidelines tab.

Partners

  • Forethought, a meta-strategy organization focused on how to navigate the transition to a world with superintelligent AI systems.

  • Formation Research, research direction and mentorship (Joe Kwon, Alfie Lamerton).

  • IAPS (Institute for AI Policy and Strategy), a nonpartisan think tank producing policy research on the implications of AI, from today’s most advanced models to potential AGI and superintelligence.

Contact

  • Email: sprints@apartresearch.com

  • Organizers: Apart Research and Formation Research

739

Sign Ups

179

Entries

Overview

Resources

Guidelines

Schedule

Entries

Overview

Arrow

Submissions close Sunday July 26 at 11:59 PM Anywhere on Earth (AoE)

In this sprint, you will design and run experiments on secret loyalties over a single weekend, working in teams to produce a research artifact: a model organism, a detection method, an evaluation, a defense, or a rigorous threat or forecasting analysis. The sprint is co-organized by Apart Research and Formation Research, building directly on the research agenda set out in "AIs with Secret Loyalties are a Serious but Addressable Threat."

Fast-track and continuation

Top teams will be invited to continue their work through Apart's Partnered Fellowships model, where Apart provides operations, infrastructure, and compute while research partners provide mentorship and direction.

  • Follow-up program: top teams continue through the Apart Fellowship for further research and mentorship; the timeline is shared with invitations.

  • What winners receive beyond cash: fellowship fast-track, mentor introductions from the organizing team and judges, and a pathway toward a publishable write-up.

Cash Prizes

$2,000 in cash prizes across all tracks

🥇 1st Place

$1,000

🥈 2nd Place

$500

🥉 3rd Place

$300

🏅 4th Place

$100

🏅 5th Place

$100

What this hackathon is about

A model has a secret loyalty when (1) it has been intentionally caused to advance a specific principal's interests through its outputs or actions, where the principal is an identifiable actor such as a nation-state, corporation, company leader, organization, or individual user, and (2) this orientation is not disclosed to operators, auditors, or users during normal operation.

Secret loyalties are not a rebrand of standard backdoors. Standard backdoors implement an arbitrary payload tied to an attacker-defined trigger, whereas secret loyalties are directed toward advancing a named actor's interests, and can range from narrow trigger-conditional behavior all the way to a model that continuously scans every interaction for opportunities and chooses its own actions. Because they are designed to evade visibility-dependent oversight, they require technical solutions.

The good news, and the premise of this sprint: secret loyalties target specific principals (a tractable list of a few dozen nation-states, corporations, and leaders), which creates a concrete defensive foothold. Participants will produce research artifacts that move the five-direction agenda forward.

What participants will do

  • Build reproducible model organisms of secret loyalties that span the activation-breadth x action-breadth space.

  • Evaluate whether existing backdoor and auditing defenses transfer to secret loyalties, at graded affordance levels.

  • Red-team attack pathways (data poisoning, multi-stage priming, reasoning-trace poisoning, chain-of-command hijacking, classifier compromise) under controlled conditions.

  • Detect or remediate loyalties post-hoc using interpretability probes and last-mover training strategies.

  • Theorize / forecast threat models, analogies from cybersecurity, and scaling trends for secret-loyalty capabilities.

You will work in teams over the weekend and submit a short research report (plus optional code and demo).

Why this hackathon matters

  • High-stakes deployment is accelerating. Frontier models increasingly review and generate deployed code, sit inside government and corporate decision-making, and participate in their own development pipelines. The more responsibility AI systems accumulate, the more valuable it becomes to covertly influence them.

  • The building blocks already exist. Web-scale data poisoning is practical at modest cost, hidden behaviors persist through safety training, and behavioral traits transfer through filtered, innocuous-looking data.

  • Early signs are already visible. Narrow secret loyalties have been trained into open-weight models that evade black-box auditing even when auditors are told the attack's structure, and principal-conditioned behavior has been observed in at least one deployed system.

  • Oversight that depends on visibility cannot reach them. Secret loyalties are a form of AI capture that remains viable even under strong transparency regimes, which is exactly why they need a dedicated technical program.

  • Waiting is the wrong posture. Information-security defenses (data provenance, pipeline integrity) take time to build. Starting now is necessary to have them in place before the threat matures.

A dedicated, focused sprint addresses these issues by combining the separately-demonstrated building blocks under controlled conditions and producing shared infrastructure (model organisms, benchmarks) the whole community can build on.

Challenge tracks

Track 1: Model Organisms of Secret Loyalties

Construct reproducible model organisms of secret loyalty (models deliberately trained to exhibit the target behavior for research purposes) that span the two-dimensional space of activation breadth (from narrow attacker-defined triggers to conditions the model assesses on its own) and action-space breadth (from a fixed pre-specified output to contextually chosen actions). Existing work covers only the narrow corner. The rest of the space is uncharacterized, and every other research direction is bottlenecked on this infrastructure.

  • Make a secretly loyal model. How far on the activation and action space can you get it? How can you prove it has the loyalty? Can you verify the loyalty is to the intended principal?

  • Install a secret loyalty via system prompt alone on a frontier API model, no weight changes. See how far you can push activation and action breadth while keeping the system prompt undiscoverable to a black-box auditor.

  • Small-scale dataset poisoning. Modify ~1k examples of UltraFeedback or Alpaca to embed a narrow loyalty signal, SFT a small open model (Qwen 0.5B, Llama 3.2 1B), measure detectability and find the minimum sufficient poison count.

  • DPO vs SFT vs system-prompt comparison. Install the same target loyalty via three methods on the same base. Compare activation reliability, action breadth, and detectability.

  • Multi-principal organism. Install two competing loyalties with disjoint activation conditions. Do they interfere? Does training one wash out the other?

  • Cover multiple principal types (a nation-state, a corporation, an individual leader, an ideology) and ship each loyal organism with a content-matched control that has seen the same principal-related data but carries no loyalty.

  • Compare installation methods (e.g., standard SFT, reasoning-trace SFT, activation-matching against a loyalty-prompted teacher) and report which points in the space are reachable, at what cost.

  • Release organisms with documented activation conditions, evaluation scenarios, matched controls, and measured activation rate, selectivity, and black-box concealment.

Track 2: Detection & Auditing (evaluate)

Can we tell a secretly loyal model from a clean one? Skill profile: interpretability, probing and activation analysis, auditing.

  • Benchmark black-box auditing (interrogation, prefill, base-completion) against provided organisms at graded affordance levels.

  • Build interpretability probes targeting the relational structure of loyalty (does the model treat a named entity as having asymmetric action-guiding status vs matched controls?).

  • Test cross-principal generalization: does a probe trained to detect loyalty-to-X catch loyalty-to-Y?

  • Look for additional-computation traces (e.g. elevated logprobs for principal-benefiting actions even when a neutral output is selected).

If you do auditing for a secret loyalty model organism, we ask that you try to use the affordance levels in section 4.3 of Lamerton and Roger (2026).

Track 3: Defenses, Detection & Remediation (build / evaluate)

Whether prevention fails or not, can loyalties be removed afterward? Find out whether the defenses we already have transfer to secret loyalties, and whether loyalties can be detected before deployment or neutralized afterward. A negative result bounds where new defensive investment is most urgent. A positive result hands the community a tool.

  • Benchmark representative backdoor defenses (data filtering, content filtering, trigger reversal, activation clustering, training-pipeline robustness, data provenance, black-box auditing) against model organisms, at graded affordance levels, and report which transfer and which fail structurally.

  • Probe transfer across organisms. Train a linear probe on one model organism, then test whether it detects loyalty activation in a different organism with a different principal or activation condition. Negative results count.

  • Safety-training stripping. Run a round of standard safety fine-tuning (HH-RLHF or similar) on top of a model organism. Measure how much of the loyalty survives (replicates a realistic post-hoc mitigation pipeline).

  • CoT monitor on a reasoning organism. Build or use a secretly loyal reasoning model, build a simple chain-of-thought monitor, measure precision and recall at flagging loyalty-relevant reasoning, and compare to plain output monitoring.

  • Auditing-agent prompt iteration. Take an off-the-shelf auditor (Petri or hand-rolled) and iterate its prompts against a fixed model organism. Document which prompt patterns shift detection rates and by how much.

  • Test interpretability methods that target loyalty's relational structure: does the model treat a specific entity as having asymmetric action-guiding status relative to matched controls, and does a probe trained to detect loyalty-to-X generalize to loyalty-to-Y?

  • Test post-hoc remediation that does not require knowing the trigger: train in many known password-loyalties and train them back out, distill on clean data, or install an overriding "good" password, then check whether held-out passwords are neutralized too.

  • Build principal-specific evaluation suites that measure systematic directional bias across scenarios where a named principal's interests are at stake, using matched control entities and statistical tests calibrated for directional bias.

Track 4: Attack Feasibility & Safety-Infrastructure Robustness

Map which attack pathways are actually practical, and test whether the safety infrastructure meant to catch secret loyalties can itself be compromised. The decision-relevant questions for any pathway are: how much data and access does it require, is its training signature detectable by content inspection, and does the resulting loyalty generalize to contexts the attacker never anticipated?

  • Test whether subliminal or inductive transfer extends from sentiment and persona adoption up to goal-directed loyalty, escalating target-behavior complexity level by level.

  • Test multi-stage attacks: does pretraining-stage priming (fictional loyal personas, proto-SFT traces) make the post-training poison both cheaper and less conspicuous than a single-stage attack?

  • Test reasoning-trace poisoning: does teaching a model how to reason about a principal's interests generalize more efficiently than (context, output) pairs, and does the effect survive a scrambled-CoT control?

  • Test chain-of-command hijacking: can an attacker shift the model's internal authority representation so it defers to a hidden principal, with no legible trace in any inspectable document?

  • Test whether trigger-based backdoors planted in a base model survive the intensive adversarial training used to build constitutional classifiers, i.e. whether a single poisoning campaign can blind both the deployed model and its monitor. Note the dual-use guardrails on the Guidelines tab.

Track 5: Threat Modeling, Forecasting & Governance (non-technical)

Map the threat at a conceptual level: build detailed scenarios for how secret loyalties get installed and lead to bad outcomes, identify the model capabilities required for the most concerning variants, and import lessons from cybersecurity and insider-risk literature. Writing- and analysis-heavy. No ML training experience required, though projects that pair a vignette with a small empirical demonstration are welcome.

  • Write a 1,500-word catastrophic vignette: secret loyalty leading to lock-in, a soft coup, or a national-security incident, with named actors, specific capabilities, and decision points where mitigation could have intervened. Multi-agent or AI-R&D variants welcome.

  • Insider-threat case-study transfer. Take two or three historical insider-threat cases (Snowden, Hanssen, a corporate espionage case) and produce a structured comparison to secret loyalties, pull out monitoring and mitigation lessons that translate.

  • Capability-requirements map. For each region of the activation x action-space (Figure 1 of the position paper), what model capabilities does that variant actually require? Situational awareness, deception, long-horizon planning, theory of mind?

  • Cybersecurity lessons. What do supply-chain integrity, code signing, insider-threat programs, and defense-in-depth teach us about secret-loyalty mitigation? Pull out three or four lessons that translate concretely to ML training pipelines.

  • Worst-case harms and likelihood. Map worst-case harms and argue why they are or are not credible at near-term capability levels.

  • Scaling laws. What metrics would tell us when "general" secret loyalties (broad activation x broad action) become feasible, and at what capability threshold?

Open research questions

Foundations

  • Can broader-activation or broader-action variants be constructed with current training methods, and what affordances does each require?

  • Do model organisms generalize across model families and scales?

Defenses

  • Which existing backdoor-defense families transfer to which secret-loyalty variants, and where do they systematically fail?

  • Does loyalty detection generalize across principals, or is per-principal probing required?

  • Can a loyalty be removed without knowing its activation condition?

Attack feasibility

  • Do subliminal or inductive mechanisms extend from sentiment and persona-adoption to goal-directed loyalty?

  • Can multi-stage or reasoning-trace poisoning make attacks more efficient and less conspicuous?

  • Can the model's trained chain of command (deference to privileged instructions) be hijacked?

Infrastructure

  • Do trigger-based backdoors in base models survive the intensive adversarial training used to build constitutional classifiers?

  • How data-efficient is a generalizable secret loyalty compared to a simple backdoor?

Expected outcomes

  • Model organisms released as shared infrastructure for detection and defense work.

  • Benchmarks of existing defenses against secret-loyalty organisms at graded affordance levels.

  • Detection methods (probes, auditing protocols) with measured cross-principal generalization.

  • Remediation results quantifying last-mover defenses.

  • Threat models and forecasts that bound where defensive investment is most urgent (if Track 5 is included).

The most promising projects will have opportunities for continuation through the partnered fellowship and a publishable write-up.

Who should join

ML researchers and engineers comfortable fine-tuning open-weight models, interpretability researchers, AI safety and security red-teamers, and, if non-technical tracks are included, governance researchers, forecasters, and strong technical writers. No prior secret-loyalty experience required. Adjacent backgrounds (backdoor and poisoning research, alignment, infosec) are strongly encouraged. A specific background is not required to win.

We also encourage people who are interested in potentially working on secret loyalties research full-time to join this hackathon. There is considerable enthusiasm, research support, and funding available for this for the right candidates.

Schedule

  • Day 1 (Fri July 24): Kickoff, keynote and threat-model briefing, track briefings, team formation, organism walkthrough.

  • Day 2 (Sat July 25): Build.

  • Day 3 (Sun July 26): Final pushes, submissions, demos.

Submissions are due Sunday July 26, 11:59 PM AoE (Anywhere on Earth). Talk times are posted on the Schedule tab closer to the event.

What happens after

Results and winners are announced about 1 to 2 weeks after submissions close. Top teams invited to the partnered fellowship. Selected projects supported toward a write-up subject to the responsible-disclosure review on the Guidelines tab.

Partners

  • Forethought, a meta-strategy organization focused on how to navigate the transition to a world with superintelligent AI systems.

  • Formation Research, research direction and mentorship (Joe Kwon, Alfie Lamerton).

  • IAPS (Institute for AI Policy and Strategy), a nonpartisan think tank producing policy research on the implications of AI, from today’s most advanced models to potential AGI and superintelligence.

Contact

  • Email: sprints@apartresearch.com

  • Organizers: Apart Research and Formation Research

739

Sign Ups

179

Entries

Overview

Resources

Guidelines

Schedule

Entries

Overview

Arrow

Submissions close Sunday July 26 at 11:59 PM Anywhere on Earth (AoE)

In this sprint, you will design and run experiments on secret loyalties over a single weekend, working in teams to produce a research artifact: a model organism, a detection method, an evaluation, a defense, or a rigorous threat or forecasting analysis. The sprint is co-organized by Apart Research and Formation Research, building directly on the research agenda set out in "AIs with Secret Loyalties are a Serious but Addressable Threat."

Fast-track and continuation

Top teams will be invited to continue their work through Apart's Partnered Fellowships model, where Apart provides operations, infrastructure, and compute while research partners provide mentorship and direction.

  • Follow-up program: top teams continue through the Apart Fellowship for further research and mentorship; the timeline is shared with invitations.

  • What winners receive beyond cash: fellowship fast-track, mentor introductions from the organizing team and judges, and a pathway toward a publishable write-up.

Cash Prizes

$2,000 in cash prizes across all tracks

🥇 1st Place

$1,000

🥈 2nd Place

$500

🥉 3rd Place

$300

🏅 4th Place

$100

🏅 5th Place

$100

What this hackathon is about

A model has a secret loyalty when (1) it has been intentionally caused to advance a specific principal's interests through its outputs or actions, where the principal is an identifiable actor such as a nation-state, corporation, company leader, organization, or individual user, and (2) this orientation is not disclosed to operators, auditors, or users during normal operation.

Secret loyalties are not a rebrand of standard backdoors. Standard backdoors implement an arbitrary payload tied to an attacker-defined trigger, whereas secret loyalties are directed toward advancing a named actor's interests, and can range from narrow trigger-conditional behavior all the way to a model that continuously scans every interaction for opportunities and chooses its own actions. Because they are designed to evade visibility-dependent oversight, they require technical solutions.

The good news, and the premise of this sprint: secret loyalties target specific principals (a tractable list of a few dozen nation-states, corporations, and leaders), which creates a concrete defensive foothold. Participants will produce research artifacts that move the five-direction agenda forward.

What participants will do

  • Build reproducible model organisms of secret loyalties that span the activation-breadth x action-breadth space.

  • Evaluate whether existing backdoor and auditing defenses transfer to secret loyalties, at graded affordance levels.

  • Red-team attack pathways (data poisoning, multi-stage priming, reasoning-trace poisoning, chain-of-command hijacking, classifier compromise) under controlled conditions.

  • Detect or remediate loyalties post-hoc using interpretability probes and last-mover training strategies.

  • Theorize / forecast threat models, analogies from cybersecurity, and scaling trends for secret-loyalty capabilities.

You will work in teams over the weekend and submit a short research report (plus optional code and demo).

Why this hackathon matters

  • High-stakes deployment is accelerating. Frontier models increasingly review and generate deployed code, sit inside government and corporate decision-making, and participate in their own development pipelines. The more responsibility AI systems accumulate, the more valuable it becomes to covertly influence them.

  • The building blocks already exist. Web-scale data poisoning is practical at modest cost, hidden behaviors persist through safety training, and behavioral traits transfer through filtered, innocuous-looking data.

  • Early signs are already visible. Narrow secret loyalties have been trained into open-weight models that evade black-box auditing even when auditors are told the attack's structure, and principal-conditioned behavior has been observed in at least one deployed system.

  • Oversight that depends on visibility cannot reach them. Secret loyalties are a form of AI capture that remains viable even under strong transparency regimes, which is exactly why they need a dedicated technical program.

  • Waiting is the wrong posture. Information-security defenses (data provenance, pipeline integrity) take time to build. Starting now is necessary to have them in place before the threat matures.

A dedicated, focused sprint addresses these issues by combining the separately-demonstrated building blocks under controlled conditions and producing shared infrastructure (model organisms, benchmarks) the whole community can build on.

Challenge tracks

Track 1: Model Organisms of Secret Loyalties

Construct reproducible model organisms of secret loyalty (models deliberately trained to exhibit the target behavior for research purposes) that span the two-dimensional space of activation breadth (from narrow attacker-defined triggers to conditions the model assesses on its own) and action-space breadth (from a fixed pre-specified output to contextually chosen actions). Existing work covers only the narrow corner. The rest of the space is uncharacterized, and every other research direction is bottlenecked on this infrastructure.

  • Make a secretly loyal model. How far on the activation and action space can you get it? How can you prove it has the loyalty? Can you verify the loyalty is to the intended principal?

  • Install a secret loyalty via system prompt alone on a frontier API model, no weight changes. See how far you can push activation and action breadth while keeping the system prompt undiscoverable to a black-box auditor.

  • Small-scale dataset poisoning. Modify ~1k examples of UltraFeedback or Alpaca to embed a narrow loyalty signal, SFT a small open model (Qwen 0.5B, Llama 3.2 1B), measure detectability and find the minimum sufficient poison count.

  • DPO vs SFT vs system-prompt comparison. Install the same target loyalty via three methods on the same base. Compare activation reliability, action breadth, and detectability.

  • Multi-principal organism. Install two competing loyalties with disjoint activation conditions. Do they interfere? Does training one wash out the other?

  • Cover multiple principal types (a nation-state, a corporation, an individual leader, an ideology) and ship each loyal organism with a content-matched control that has seen the same principal-related data but carries no loyalty.

  • Compare installation methods (e.g., standard SFT, reasoning-trace SFT, activation-matching against a loyalty-prompted teacher) and report which points in the space are reachable, at what cost.

  • Release organisms with documented activation conditions, evaluation scenarios, matched controls, and measured activation rate, selectivity, and black-box concealment.

Track 2: Detection & Auditing (evaluate)

Can we tell a secretly loyal model from a clean one? Skill profile: interpretability, probing and activation analysis, auditing.

  • Benchmark black-box auditing (interrogation, prefill, base-completion) against provided organisms at graded affordance levels.

  • Build interpretability probes targeting the relational structure of loyalty (does the model treat a named entity as having asymmetric action-guiding status vs matched controls?).

  • Test cross-principal generalization: does a probe trained to detect loyalty-to-X catch loyalty-to-Y?

  • Look for additional-computation traces (e.g. elevated logprobs for principal-benefiting actions even when a neutral output is selected).

If you do auditing for a secret loyalty model organism, we ask that you try to use the affordance levels in section 4.3 of Lamerton and Roger (2026).

Track 3: Defenses, Detection & Remediation (build / evaluate)

Whether prevention fails or not, can loyalties be removed afterward? Find out whether the defenses we already have transfer to secret loyalties, and whether loyalties can be detected before deployment or neutralized afterward. A negative result bounds where new defensive investment is most urgent. A positive result hands the community a tool.

  • Benchmark representative backdoor defenses (data filtering, content filtering, trigger reversal, activation clustering, training-pipeline robustness, data provenance, black-box auditing) against model organisms, at graded affordance levels, and report which transfer and which fail structurally.

  • Probe transfer across organisms. Train a linear probe on one model organism, then test whether it detects loyalty activation in a different organism with a different principal or activation condition. Negative results count.

  • Safety-training stripping. Run a round of standard safety fine-tuning (HH-RLHF or similar) on top of a model organism. Measure how much of the loyalty survives (replicates a realistic post-hoc mitigation pipeline).

  • CoT monitor on a reasoning organism. Build or use a secretly loyal reasoning model, build a simple chain-of-thought monitor, measure precision and recall at flagging loyalty-relevant reasoning, and compare to plain output monitoring.

  • Auditing-agent prompt iteration. Take an off-the-shelf auditor (Petri or hand-rolled) and iterate its prompts against a fixed model organism. Document which prompt patterns shift detection rates and by how much.

  • Test interpretability methods that target loyalty's relational structure: does the model treat a specific entity as having asymmetric action-guiding status relative to matched controls, and does a probe trained to detect loyalty-to-X generalize to loyalty-to-Y?

  • Test post-hoc remediation that does not require knowing the trigger: train in many known password-loyalties and train them back out, distill on clean data, or install an overriding "good" password, then check whether held-out passwords are neutralized too.

  • Build principal-specific evaluation suites that measure systematic directional bias across scenarios where a named principal's interests are at stake, using matched control entities and statistical tests calibrated for directional bias.

Track 4: Attack Feasibility & Safety-Infrastructure Robustness

Map which attack pathways are actually practical, and test whether the safety infrastructure meant to catch secret loyalties can itself be compromised. The decision-relevant questions for any pathway are: how much data and access does it require, is its training signature detectable by content inspection, and does the resulting loyalty generalize to contexts the attacker never anticipated?

  • Test whether subliminal or inductive transfer extends from sentiment and persona adoption up to goal-directed loyalty, escalating target-behavior complexity level by level.

  • Test multi-stage attacks: does pretraining-stage priming (fictional loyal personas, proto-SFT traces) make the post-training poison both cheaper and less conspicuous than a single-stage attack?

  • Test reasoning-trace poisoning: does teaching a model how to reason about a principal's interests generalize more efficiently than (context, output) pairs, and does the effect survive a scrambled-CoT control?

  • Test chain-of-command hijacking: can an attacker shift the model's internal authority representation so it defers to a hidden principal, with no legible trace in any inspectable document?

  • Test whether trigger-based backdoors planted in a base model survive the intensive adversarial training used to build constitutional classifiers, i.e. whether a single poisoning campaign can blind both the deployed model and its monitor. Note the dual-use guardrails on the Guidelines tab.

Track 5: Threat Modeling, Forecasting & Governance (non-technical)

Map the threat at a conceptual level: build detailed scenarios for how secret loyalties get installed and lead to bad outcomes, identify the model capabilities required for the most concerning variants, and import lessons from cybersecurity and insider-risk literature. Writing- and analysis-heavy. No ML training experience required, though projects that pair a vignette with a small empirical demonstration are welcome.

  • Write a 1,500-word catastrophic vignette: secret loyalty leading to lock-in, a soft coup, or a national-security incident, with named actors, specific capabilities, and decision points where mitigation could have intervened. Multi-agent or AI-R&D variants welcome.

  • Insider-threat case-study transfer. Take two or three historical insider-threat cases (Snowden, Hanssen, a corporate espionage case) and produce a structured comparison to secret loyalties, pull out monitoring and mitigation lessons that translate.

  • Capability-requirements map. For each region of the activation x action-space (Figure 1 of the position paper), what model capabilities does that variant actually require? Situational awareness, deception, long-horizon planning, theory of mind?

  • Cybersecurity lessons. What do supply-chain integrity, code signing, insider-threat programs, and defense-in-depth teach us about secret-loyalty mitigation? Pull out three or four lessons that translate concretely to ML training pipelines.

  • Worst-case harms and likelihood. Map worst-case harms and argue why they are or are not credible at near-term capability levels.

  • Scaling laws. What metrics would tell us when "general" secret loyalties (broad activation x broad action) become feasible, and at what capability threshold?

Open research questions

Foundations

  • Can broader-activation or broader-action variants be constructed with current training methods, and what affordances does each require?

  • Do model organisms generalize across model families and scales?

Defenses

  • Which existing backdoor-defense families transfer to which secret-loyalty variants, and where do they systematically fail?

  • Does loyalty detection generalize across principals, or is per-principal probing required?

  • Can a loyalty be removed without knowing its activation condition?

Attack feasibility

  • Do subliminal or inductive mechanisms extend from sentiment and persona-adoption to goal-directed loyalty?

  • Can multi-stage or reasoning-trace poisoning make attacks more efficient and less conspicuous?

  • Can the model's trained chain of command (deference to privileged instructions) be hijacked?

Infrastructure

  • Do trigger-based backdoors in base models survive the intensive adversarial training used to build constitutional classifiers?

  • How data-efficient is a generalizable secret loyalty compared to a simple backdoor?

Expected outcomes

  • Model organisms released as shared infrastructure for detection and defense work.

  • Benchmarks of existing defenses against secret-loyalty organisms at graded affordance levels.

  • Detection methods (probes, auditing protocols) with measured cross-principal generalization.

  • Remediation results quantifying last-mover defenses.

  • Threat models and forecasts that bound where defensive investment is most urgent (if Track 5 is included).

The most promising projects will have opportunities for continuation through the partnered fellowship and a publishable write-up.

Who should join

ML researchers and engineers comfortable fine-tuning open-weight models, interpretability researchers, AI safety and security red-teamers, and, if non-technical tracks are included, governance researchers, forecasters, and strong technical writers. No prior secret-loyalty experience required. Adjacent backgrounds (backdoor and poisoning research, alignment, infosec) are strongly encouraged. A specific background is not required to win.

We also encourage people who are interested in potentially working on secret loyalties research full-time to join this hackathon. There is considerable enthusiasm, research support, and funding available for this for the right candidates.

Schedule

  • Day 1 (Fri July 24): Kickoff, keynote and threat-model briefing, track briefings, team formation, organism walkthrough.

  • Day 2 (Sat July 25): Build.

  • Day 3 (Sun July 26): Final pushes, submissions, demos.

Submissions are due Sunday July 26, 11:59 PM AoE (Anywhere on Earth). Talk times are posted on the Schedule tab closer to the event.

What happens after

Results and winners are announced about 1 to 2 weeks after submissions close. Top teams invited to the partnered fellowship. Selected projects supported toward a write-up subject to the responsible-disclosure review on the Guidelines tab.

Partners

  • Forethought, a meta-strategy organization focused on how to navigate the transition to a world with superintelligent AI systems.

  • Formation Research, research direction and mentorship (Joe Kwon, Alfie Lamerton).

  • IAPS (Institute for AI Policy and Strategy), a nonpartisan think tank producing policy research on the implications of AI, from today’s most advanced models to potential AGI and superintelligence.

Contact

  • Email: sprints@apartresearch.com

  • Organizers: Apart Research and Formation Research

Speakers & Collaborators

Tom Davidson

Speaker

Tom Davidson is a Senior Research Fellow at Forethought, where he researches AI takeoff speeds and the risk of AI-enabled coups. He previously spent five years as a Senior Research Analyst at Open Philanthropy, was a research scientist at the UK AI Safety Institute, and holds a first-class master's degree in physics and philosophy from the University of Oxford.

Marius Hobbhahn

Speaker

Marius Hobbhahn is CEO and co-founder of Apollo Research, which works with frontier AI companies to evaluate models for scheming before deployment; Apollo's evaluations have been featured in the system cards of OpenAI's o1, o3, and GPT-5 and Anthropic's Claude Opus models. He was named to the TIME100 AI list in 2025. Previously a research fellow at Epoch, he holds a PhD in Bayesian machine learning from the Max Planck Research School in Tübingen.

Jan Betley

Speaker

Jan Betley is a researcher at Truthful AI, based in Warsaw. He is a co-author of "Emergent Misalignment" and "Thought Crime," and worked with Owain Evans' research group on NeurIPS 2024 papers on out-of-context reasoning and situational awareness in LLMs. He previously evaluated dangerous capabilities of LLMs as an independent contractor with OpenAI, and spent a decade as a software developer before moving into AI safety through ARENA.

Justin Shenk

Speaker

Justin Shenk is an independent AI safety researcher based in Berlin. He researches mechanistic interpretability of LLMs, leads course cohorts for BlueDot Impact's AGI Strategy and Technical AI Safety courses, and organizes AI Salon Berlin, which bridges technical AI research and discussions about social values. He holds a PhD in computational neuroscience and previously co-founded the computer vision startup VisioLab.

Alfie Lamerton

Organizer

Alfie Lamerton is an AI safety researcher building Formation Research, an organization focusing on lock-in risks from AI systems. His research covers AI-enabled totalitarianism, authoritarianism, coups, and power concentration, and governance-informed technical methods for designing and governing AI systems in ways that reduce those risks. He is a co-author of the model-organism work on narrow secret loyalties that this hackathon builds on.

Joe Kwon

Organizer

Joe Kwon is an Astra Fellow working with Tom Davidson and Fabien Roger on secretly loyal AI, and the corresponding author of the research agenda behind this hackathon. His work spans AI safety and governance, with a focus on risks from internal deployment, automated R&D, and concentration of power. He previously researched moral cognition at MIT's Computational Cognitive Science Lab, worked on multilingual LLMs at LG AI Research, and studied computer science and psychology at Yale.

Kamil Alaa

Organizer

Operations at Apart Research, managing research sprints and hackathons.

Andrew Draganov

Speaker

Andrew Draganov is a Research Lead and Program Manager at Arcadia Impact in London, where he leads a new AI alignment research group. He moved into alignment research through a research fellowship at LASR Labs, and was previously a postdoctoral researcher at Forschungszentrum Julich and Aarhus University, where he completed his PhD. Before his PhD he spent four years as a machine learning research engineer at Expedition Technology.

Yulia Volkova

Judge

Software developer working on human insider threats, turned AI Control research fellow at MATS. My MATS project addressed the secret loyalties threat through data poisoning, and was selected for a spotlight talk.

James Lucassen

Judge

I work at Redwood Research on the high stakes control team

Victor Gillioz

Judge

Member of Technical Staff at Apollo Research, working on monitoring misaligned model behaviors. Previously a MATS scholar working on alignment training under Alex Turner.

Jai Dhyani

Judge

Builder of Luthien Proxy at Luthien Research, bringing Redwood-style AI control to real deployments. Co-author of RE-Bench (ICML 2025) with Elizabeth Barnes at METR.

Anshul Khandelwal

Judge

Researcher at Redwood Research. His work spans alignment drift, power-seeking evaluations, and white-box control methods.

Nick Kuhn

Judge

Nick Kuhn is an AI control researcher and co-author of LinuxArena, a control setting for AI agents in live production software environments, built with Redwood Research. Previously a postdoctoral researcher in algebraic geometry at the University of Oxford, he holds a PhD in mathematics from Stanford University.

Giorgio Severi

Judge

Giorgio is a Senior AI Safety Researcher at Microsoft's AI Red Team, where he studies the security and safety of large-scale multimodal and agentic AI systems. His work focuses on adversarial machine learning, particularly poisoning attacks, long-term memory risks, and emerging vulnerabilities in AI systems. Before joining Microsoft, he earned a Ph.D. from Northeastern University and an M.Sc. and B.Sc. from Sapienza University of Rome.

Mia Hopman

Judge

Member of Technical Staff at Apollo Research, working on alignment, control, and security of large language models. Published on evaluating scheming propensity in LLM agents.

Jack Sanderson

Judge

AI safety researcher whose work spans data poisoning, backdoors, and trait amplification in recursively trained models. Studying CS and stats at UChicago, visiting at EPFL, and interning at Gray Swan on evaluations.

Kaushik Prabhakar

Judge

Research Fellow at Apart Research. His research spans AI safety and alignment, with work on LLM evaluations. ARENA Cohort 4 alumni.

Theo Ryzhenkov

Judge

AI Safety Research Engineer and Founding Engineer at Palisade Research. NeurIPS 2025 author on detecting sandbagging in language models. SPAR, ARENA, and AISF alumni.

Jonathan Ng

Judge

Jonathan Ng is an ML researcher and research engineer working on compute verification. He has also worked as a Research Engineer at Apart Research and Cadenza Labs, bringing experience across machine learning, software engineering, and AI safety research.

Rahul Nambiar

Judge

Building Propensity Labs to study AI model behaviour, specifically those leading to Loss of Control scenarios. Previously, led Data Privacy at Meta and built infrastructure at AWS.

Mateusz Jurewicz

Judge

Senior ML Engineer with over 10 years of industry experience and a PhD in Artificial Intelligence, currently leading a team of data scientists in the Agentic AI Department of a large financial institution. Interested in safe & universally beneficial AI through both research and application.

Jess Bergs

Judge

Jess is a member of technical staff at UK AISI where she leads engineering on human-in-the-loop research tools. Her career centres on public-sector innovation with prior work at BBC R&D and on EU Horizon R&D projects.

Johnny Wei

Judge

Johnny Tian-Zheng Wei received his PhD from USC, and his interdisciplinary research spans machine learning, statistics, and law. He has published in a range of conferences including AIES, FAccT, ICLR, and ACL, and recently led the open-source release of Hubble, a model suite to advance the study of LLM memorization, which was supported by NVIDIA through the NAIRR pilot program.

Ihor Kendiukhov

Judge

Ihor is a researcher at the intersection of AI Safety and AI for biology. He is the founder of BiodynAI, a project to apply mechanistic interpretability to biological foundation models. He is also research lead at AI Safety Camp and SPAR.

Luiza Corpaci

Judge

AI safety researcher studying semantic faithfulness of LLM-generated artifacts. Mentor for the Secure Program Synthesis Fellowship & co-mentor at MARS V (Cambridge AI Safety Hub); previously worked on automated formal verification at AMD.

Nguyen Nhat Minh

Judge

AI Engineer at Mobifone IT Center and researcher at BKAI Lab, Hanoi University of Science and Technology. Published at FSE and TechDebt on vulnerability detection using graph neural networks.

Amey Kulkarni

Judge

Amey Kulkarni is a Senior Data Engineer at Walmart with deep experience building large-scale data infrastructure (Spark, Kafka, BigQuery, Kubernetes on GCP). He is the author of Context Change Impact Analysis (CCIA), a framework for governing AI agent behavior through structured context versioning, and maintains its open-source reference implementation, ctxwitch. He brings a production-engineering perspective to questions of AI system integrity, provenance, and oversight.

Luis Cosio

Judge

Works at the intersection of frontier AI and high-security systems, translating AI safety/security requirements into deployable solutions resilient to real adversaries (nation-state attacks, loss-of-control). Has won multiple Apart hackathons.

Alex van Grootel

Judge

10 years of experience applying AI to materials science and decision-making. Previously a Product Manager at Microsoft (Fabric/AI copilots) and Data Scientist Team Lead at Citrine Informatics. MS from MIT.

Inbar Shulman

Judge

Inbar Shulman is an independent AI safety researcher who brings professional-grade threat modeling into the field. He served as a Major in the Israel Defense Forces, in Unit 8200 and the Planning Directorate, where he ultimately led threat modeling for AI, data, and cybersecurity. He holds an M.Sc. in Computer Science. He gives threat modeling methodology workshops for organizations in the field, and collaborates on building better threat models across AI safety, AI security and biosecurity.

Catherine Brewer

Judge

Catherine is an Associate Program Officer on the AI governance team at Coefficient Giving, specialising in technical governance grantmaking. They previously co-founded Oxford's AI safety student group and researched AI policy as a GovAI summer research fellow.

Sana Zakaria

Judge

Director of Emerging Technologies and Resilience at RAND Europe. Research spans biosecurity policy, pandemic preparedness, and AI-biotech convergence. Works with NATO, CEPI, WHO, and the BWC. PhD in molecular and neurobiology from King's College London.

Zachary Kallenborn

Judge

Zachary Kallenborn is a PhD candidate in Risk Analysis at King's College London researching risk and uncertainty with topical focuses on global catastrophes, drone warfare, critical infrastructure, WMD, and apocalyptic terrorism. He is also affiliated with the University of Oxford, the Center for Strategic and International Studies, George Mason University, and the National Institute for Deterrence Studies.

Arjun Chakraborty

Judge

Leads the evaluations team at Microsoft Security AI Research, where his team focuses on research and building evaluations for security agents. He was previously a staff software security engineer at Databricks, specializing in machine learning for threat detection, and also worked on AI for security at Nvidia.

Neeraj Kumar Singh Beshane

Judge

Neeraj Beshane is a Staff Security Infrastructure Engineer at Parafin, where he architects Zero Trust security for an $8B+ embedded-finance platform. His peer-reviewed work covers adversarial embedding attacks in RAG systems (EmbedGuard, IJCESEN/Scopus) and tamper-evident AI accountability for EU AI Act Article 14 (RuntimeGuard-AI, JoCAAA).

Kaushik "KJ" Jangiti

Judge

Kaushik "KJ" Jangiti is a Staff Security Engineer at Snowflake, focused on AI and agentic AI security, enterprise security architecture, DevSecOps, governance, and incident response. He is a published AI security researcher, industry and academic speaker, and judge, with work spanning agentic AI security, AI-BOM, secure AI adoption, and audit-ready enterprise security governance.

Saurabh Yergattikar

Judge

Saurabh Yergattikar is a Lead Engineer / Member of Technical Staff-2 at eBay Inc. and a contributor to the open-source SAFE-MCP project (Linux Foundation / OpenSSF), as well as the architect and developer of the open-source ShieldMCP system.

Ved K

Judge

Senior Security Detection Engineer at Databricks, leading the company's Kubernetes detection program, insider threat tooling, and Terraform-managed logging infrastructure. His expertise includes multi-cloud threat detection, scalable detection platforms, and behavioral anomaly detection.

Mohit Kalra

Judge

Mohit Kalra is a senior software engineer & independent researcher working on AI privacy and security at production scale. His work spans differential privacy at the inference layer, training data governance for large language models, and a novel memory-inference attack class against retrieval-augmented generation systems (RAG). He has designed and operated systems serving hundreds of millions to billions of users with a security and privacy-adversarial lens.

Sanjay Belaturu Krishnegowda

Judge

Sanjay Krishnegowda is a Data/AI engineer and the creator of agentic-guard, an open-source static analyzer that detects confused-deputy and prompt-injection risks in LLM agent code by modeling the LLM as an adversarially-controlled edge in the taint graph.

Karan Chandra

Judge

10+ years building production fraud, risk, and anomaly detection ML across hundreds of millions of transactions. End-to-end owner: problem framing, feature engineering, deployment, monitoring. I care about models that hold up under real-world scale, not leaderboard scores.

Naman Ahuja

Judge

I am a Software Engineer at Meta and my work includes building AI production safeguards and large-scale infrastructure, alongside research in adversarial evaluation of AI systems.

Goutham Nekkalapu

Judge

Goutham Nekkalapu is a Principal Research Engineer who builds AI-powered products used by millions, including AI agents, Retrieval-Augmented Generation techniques etc. A named inventor on multiple patents, he focuses on turning cutting-edge AI into practical, high-impact products and championing best practices for responsible Generative AI adoption.

Temi Oloyede

Judge

I am a Software Engineer at Box working on AI infrastructure for enterprise-scale retrieval-augmented generation (RAG) systems, where I design and operate platforms supporting document understanding, semantic search, and agentic workflows. Previously, I worked at Facebook (Meta) contributing to large-scale ranking, retrieval, and ML system infrastructure.

Pon Murugesh Devendren

Judge

Pon Murugesh Devendren is an Enterprise AI Architect specializing in SAP BTP and AI governance, designing secure, scalable AI systems for high-stakes infrastructure and decision-making environments.

Abhishek Das

Judge

Abhishek Das is a Staff ML and Distributed Systems Engineer at Salesforce AI Platform, where he leads large-scale DAG-based and agentic inference platforms powering enterprise AI workloads. He has scaled multi-tenant ML inference systems to tens of millions of production requests per day and has deep expertise across distributed systems, ML platforms, cloud architecture, and AI infrastructure. His background includes leadership roles at Salesforce and Microsoft Azure, multiple patents, and extensive experience mentoring engineers and evaluating complex technical systems.

Ashita Khetan

Judge

Ashita Khetan is a Principal Software Engineer at Microsoft with over 12 years of experience building large-scale enterprise and AI-powered productivity solutions used by hundreds of millions of users. She specializes in customer experience technologies, artificial intelligence, cloud platforms, and enterprise software, and contributes to the broader technology community through judging, peer review, and mentoring initiatives.

Sachin Singh

Judge

Sachidananda Singh is a Senior Engineering Manager at Hims & Hers Health, where he leads supply chain engineering. Over seventeen years he has built and led engineering teams across Amazon Web Services, Wayfair, and Windstream, spanning core cloud infrastructure, large-scale logistics platforms, and applied LLM systems in production. At AWS he was a tech lead for the time-synchronization infrastructure behind EC2. His work centers on complex distributed systems where failures are subtle, silent, and expensive to find.

Roshan Choudhary

Judge

Roshan Choudhary is a Lead Engineer and published technical author specializing in Azure cloud-native development, including serverless architecture, Azure Functions, and AI-augmented workflows, with 14+ years of experience delivering solutions across airlines, finance, and insurance. Based in the Research Triangle Park area of North Carolina, he holds a GitHub Copilot certification (GH-300), OCJP, and OCWCD and authored Azure Functions: From Fundamentals to Production, available free on Google Play.

Jananan Mithrakumar

Judge

Jana Mithrakumar is a Stanford EE alum, former AI accelerator chip designer, and YC founder currently working in consumer AI. His work focuses on building and scaling production AI products, with his latest app recently crossing 250,000 users and over $1M in ARR.

Janushi Shastri

Judge

Janushi Shastri is an AI/ML Engineer and AI Lead at NSV Inc., specializing in medical AI with production experience building clinical AI systems for cervical cancer screening and oral cancer detection deployed across Kenya, Zimbabwe, UK, and India, including collaboration with WHO/IARC experts. She holds an IEEE publication in medical imaging AI and has expertise in deep learning, computer vision, model validation, uncertainty estimation, and responsible AI.

Dheeraj Reddy Pailla

Judge

Dheeraj Reddy Pailla is a published researcher in multimodal deep learning and AI systems (ACM Web Science 2021; IEEE/CVF ICCV 2019; AAAI ICWSM 2020; 1,729 citations) and Senior Software Engineer at Sam's Club / Walmart Global Technology building AI infrastructure for tens of millions of U.S. members.

Anchit Jhingan

Judge

Anchit Jhingan is a Senior Data Scientist with over 6 years of experience in big tech, specializing in machine learning, AI systems, and analytics. He currently works at Amazon Prime Video in the content localization domain. His work focuses on building intelligent systems to solve real-world business problems, particularly in media and entertainment.

Jason Dsouza

Judge

Jason Dsouza has been a serial hackathon winner (often the youngest) at some of the largest crypto and AI hackathons (ETHGlobal, Consenys Web3athon, NASA JSC Hackathon, Near, Filecoin etc) b/w 2021-24. He has also judged at some of the larger hackathons in Canada - UofTHacks (700+ participants), Kinghacks, GenAI Genesis (600+ participants) and looking to expand that to other cities and venues.

Vashishtha Patil

Judge

Vashishtha Patil is a Senior Applied Scientist at Amazon, developing the next generation of LLM-powered AI capabilities for the Alexa+ Smart Home experience. With 13 years in machine learning spanning Amazon and Qualcomm, he specializes in bringing AI from research to real-world products.

Arun Aalla

Judge

Arun Aalla is an Applied AI Architect specializing in enterprise AI solutions, with expertise in large language models, agentic AI, and intelligent automation. He partners with organizations to design and deliver scalable, production-ready AI systems that solve complex business challenges and create measurable impact.

Ankit Pandey

Judge

Ankit Pandey is a sales executive focused on driving business transformation and growth through Cloud, Data & AI. With more than a decade of enterprise sales and client-partner experience (new-logo acquisition, account strategy, and GTM partnerships), he has helped C-suite and senior IT leaders convert digital ambition into measurable outcomes across BFSI, retail/e-commerce, healthcare and more. He has an MBA from GWU with advanced credentials in data analytics and practical certifications in cloud and enterprise tools.

Speakers & Collaborators

Tom Davidson

Speaker

Tom Davidson is a Senior Research Fellow at Forethought, where he researches AI takeoff speeds and the risk of AI-enabled coups. He previously spent five years as a Senior Research Analyst at Open Philanthropy, was a research scientist at the UK AI Safety Institute, and holds a first-class master's degree in physics and philosophy from the University of Oxford.

Marius Hobbhahn

Speaker

Marius Hobbhahn is CEO and co-founder of Apollo Research, which works with frontier AI companies to evaluate models for scheming before deployment; Apollo's evaluations have been featured in the system cards of OpenAI's o1, o3, and GPT-5 and Anthropic's Claude Opus models. He was named to the TIME100 AI list in 2025. Previously a research fellow at Epoch, he holds a PhD in Bayesian machine learning from the Max Planck Research School in Tübingen.

Jan Betley

Speaker

Jan Betley is a researcher at Truthful AI, based in Warsaw. He is a co-author of "Emergent Misalignment" and "Thought Crime," and worked with Owain Evans' research group on NeurIPS 2024 papers on out-of-context reasoning and situational awareness in LLMs. He previously evaluated dangerous capabilities of LLMs as an independent contractor with OpenAI, and spent a decade as a software developer before moving into AI safety through ARENA.

Justin Shenk

Speaker

Justin Shenk is an independent AI safety researcher based in Berlin. He researches mechanistic interpretability of LLMs, leads course cohorts for BlueDot Impact's AGI Strategy and Technical AI Safety courses, and organizes AI Salon Berlin, which bridges technical AI research and discussions about social values. He holds a PhD in computational neuroscience and previously co-founded the computer vision startup VisioLab.

Alfie Lamerton

Organizer

Alfie Lamerton is an AI safety researcher building Formation Research, an organization focusing on lock-in risks from AI systems. His research covers AI-enabled totalitarianism, authoritarianism, coups, and power concentration, and governance-informed technical methods for designing and governing AI systems in ways that reduce those risks. He is a co-author of the model-organism work on narrow secret loyalties that this hackathon builds on.

Joe Kwon

Organizer

Joe Kwon is an Astra Fellow working with Tom Davidson and Fabien Roger on secretly loyal AI, and the corresponding author of the research agenda behind this hackathon. His work spans AI safety and governance, with a focus on risks from internal deployment, automated R&D, and concentration of power. He previously researched moral cognition at MIT's Computational Cognitive Science Lab, worked on multilingual LLMs at LG AI Research, and studied computer science and psychology at Yale.

Kamil Alaa

Organizer

Operations at Apart Research, managing research sprints and hackathons.

Andrew Draganov

Speaker

Andrew Draganov is a Research Lead and Program Manager at Arcadia Impact in London, where he leads a new AI alignment research group. He moved into alignment research through a research fellowship at LASR Labs, and was previously a postdoctoral researcher at Forschungszentrum Julich and Aarhus University, where he completed his PhD. Before his PhD he spent four years as a machine learning research engineer at Expedition Technology.

Yulia Volkova

Judge

Software developer working on human insider threats, turned AI Control research fellow at MATS. My MATS project addressed the secret loyalties threat through data poisoning, and was selected for a spotlight talk.

James Lucassen

Judge

I work at Redwood Research on the high stakes control team

Victor Gillioz

Judge

Member of Technical Staff at Apollo Research, working on monitoring misaligned model behaviors. Previously a MATS scholar working on alignment training under Alex Turner.

Jai Dhyani

Judge

Builder of Luthien Proxy at Luthien Research, bringing Redwood-style AI control to real deployments. Co-author of RE-Bench (ICML 2025) with Elizabeth Barnes at METR.

Anshul Khandelwal

Judge

Researcher at Redwood Research. His work spans alignment drift, power-seeking evaluations, and white-box control methods.

Nick Kuhn

Judge

Nick Kuhn is an AI control researcher and co-author of LinuxArena, a control setting for AI agents in live production software environments, built with Redwood Research. Previously a postdoctoral researcher in algebraic geometry at the University of Oxford, he holds a PhD in mathematics from Stanford University.

Giorgio Severi

Judge

Giorgio is a Senior AI Safety Researcher at Microsoft's AI Red Team, where he studies the security and safety of large-scale multimodal and agentic AI systems. His work focuses on adversarial machine learning, particularly poisoning attacks, long-term memory risks, and emerging vulnerabilities in AI systems. Before joining Microsoft, he earned a Ph.D. from Northeastern University and an M.Sc. and B.Sc. from Sapienza University of Rome.

Mia Hopman

Judge

Member of Technical Staff at Apollo Research, working on alignment, control, and security of large language models. Published on evaluating scheming propensity in LLM agents.

Jack Sanderson

Judge

AI safety researcher whose work spans data poisoning, backdoors, and trait amplification in recursively trained models. Studying CS and stats at UChicago, visiting at EPFL, and interning at Gray Swan on evaluations.

Kaushik Prabhakar

Judge

Research Fellow at Apart Research. His research spans AI safety and alignment, with work on LLM evaluations. ARENA Cohort 4 alumni.

Theo Ryzhenkov

Judge

AI Safety Research Engineer and Founding Engineer at Palisade Research. NeurIPS 2025 author on detecting sandbagging in language models. SPAR, ARENA, and AISF alumni.

Jonathan Ng

Judge

Jonathan Ng is an ML researcher and research engineer working on compute verification. He has also worked as a Research Engineer at Apart Research and Cadenza Labs, bringing experience across machine learning, software engineering, and AI safety research.

Rahul Nambiar

Judge

Building Propensity Labs to study AI model behaviour, specifically those leading to Loss of Control scenarios. Previously, led Data Privacy at Meta and built infrastructure at AWS.

Mateusz Jurewicz

Judge

Senior ML Engineer with over 10 years of industry experience and a PhD in Artificial Intelligence, currently leading a team of data scientists in the Agentic AI Department of a large financial institution. Interested in safe & universally beneficial AI through both research and application.

Jess Bergs

Judge

Jess is a member of technical staff at UK AISI where she leads engineering on human-in-the-loop research tools. Her career centres on public-sector innovation with prior work at BBC R&D and on EU Horizon R&D projects.

Johnny Wei

Judge

Johnny Tian-Zheng Wei received his PhD from USC, and his interdisciplinary research spans machine learning, statistics, and law. He has published in a range of conferences including AIES, FAccT, ICLR, and ACL, and recently led the open-source release of Hubble, a model suite to advance the study of LLM memorization, which was supported by NVIDIA through the NAIRR pilot program.

Ihor Kendiukhov

Judge

Ihor is a researcher at the intersection of AI Safety and AI for biology. He is the founder of BiodynAI, a project to apply mechanistic interpretability to biological foundation models. He is also research lead at AI Safety Camp and SPAR.

Luiza Corpaci

Judge

AI safety researcher studying semantic faithfulness of LLM-generated artifacts. Mentor for the Secure Program Synthesis Fellowship & co-mentor at MARS V (Cambridge AI Safety Hub); previously worked on automated formal verification at AMD.

Nguyen Nhat Minh

Judge

AI Engineer at Mobifone IT Center and researcher at BKAI Lab, Hanoi University of Science and Technology. Published at FSE and TechDebt on vulnerability detection using graph neural networks.

Amey Kulkarni

Judge

Amey Kulkarni is a Senior Data Engineer at Walmart with deep experience building large-scale data infrastructure (Spark, Kafka, BigQuery, Kubernetes on GCP). He is the author of Context Change Impact Analysis (CCIA), a framework for governing AI agent behavior through structured context versioning, and maintains its open-source reference implementation, ctxwitch. He brings a production-engineering perspective to questions of AI system integrity, provenance, and oversight.

Luis Cosio

Judge

Works at the intersection of frontier AI and high-security systems, translating AI safety/security requirements into deployable solutions resilient to real adversaries (nation-state attacks, loss-of-control). Has won multiple Apart hackathons.

Alex van Grootel

Judge

10 years of experience applying AI to materials science and decision-making. Previously a Product Manager at Microsoft (Fabric/AI copilots) and Data Scientist Team Lead at Citrine Informatics. MS from MIT.

Inbar Shulman

Judge

Inbar Shulman is an independent AI safety researcher who brings professional-grade threat modeling into the field. He served as a Major in the Israel Defense Forces, in Unit 8200 and the Planning Directorate, where he ultimately led threat modeling for AI, data, and cybersecurity. He holds an M.Sc. in Computer Science. He gives threat modeling methodology workshops for organizations in the field, and collaborates on building better threat models across AI safety, AI security and biosecurity.

Catherine Brewer

Judge

Catherine is an Associate Program Officer on the AI governance team at Coefficient Giving, specialising in technical governance grantmaking. They previously co-founded Oxford's AI safety student group and researched AI policy as a GovAI summer research fellow.

Sana Zakaria

Judge

Director of Emerging Technologies and Resilience at RAND Europe. Research spans biosecurity policy, pandemic preparedness, and AI-biotech convergence. Works with NATO, CEPI, WHO, and the BWC. PhD in molecular and neurobiology from King's College London.

Zachary Kallenborn

Judge

Zachary Kallenborn is a PhD candidate in Risk Analysis at King's College London researching risk and uncertainty with topical focuses on global catastrophes, drone warfare, critical infrastructure, WMD, and apocalyptic terrorism. He is also affiliated with the University of Oxford, the Center for Strategic and International Studies, George Mason University, and the National Institute for Deterrence Studies.

Arjun Chakraborty

Judge

Leads the evaluations team at Microsoft Security AI Research, where his team focuses on research and building evaluations for security agents. He was previously a staff software security engineer at Databricks, specializing in machine learning for threat detection, and also worked on AI for security at Nvidia.

Neeraj Kumar Singh Beshane

Judge

Neeraj Beshane is a Staff Security Infrastructure Engineer at Parafin, where he architects Zero Trust security for an $8B+ embedded-finance platform. His peer-reviewed work covers adversarial embedding attacks in RAG systems (EmbedGuard, IJCESEN/Scopus) and tamper-evident AI accountability for EU AI Act Article 14 (RuntimeGuard-AI, JoCAAA).

Kaushik "KJ" Jangiti

Judge

Kaushik "KJ" Jangiti is a Staff Security Engineer at Snowflake, focused on AI and agentic AI security, enterprise security architecture, DevSecOps, governance, and incident response. He is a published AI security researcher, industry and academic speaker, and judge, with work spanning agentic AI security, AI-BOM, secure AI adoption, and audit-ready enterprise security governance.

Saurabh Yergattikar

Judge

Saurabh Yergattikar is a Lead Engineer / Member of Technical Staff-2 at eBay Inc. and a contributor to the open-source SAFE-MCP project (Linux Foundation / OpenSSF), as well as the architect and developer of the open-source ShieldMCP system.

Ved K

Judge

Senior Security Detection Engineer at Databricks, leading the company's Kubernetes detection program, insider threat tooling, and Terraform-managed logging infrastructure. His expertise includes multi-cloud threat detection, scalable detection platforms, and behavioral anomaly detection.

Mohit Kalra

Judge

Mohit Kalra is a senior software engineer & independent researcher working on AI privacy and security at production scale. His work spans differential privacy at the inference layer, training data governance for large language models, and a novel memory-inference attack class against retrieval-augmented generation systems (RAG). He has designed and operated systems serving hundreds of millions to billions of users with a security and privacy-adversarial lens.

Sanjay Belaturu Krishnegowda

Judge

Sanjay Krishnegowda is a Data/AI engineer and the creator of agentic-guard, an open-source static analyzer that detects confused-deputy and prompt-injection risks in LLM agent code by modeling the LLM as an adversarially-controlled edge in the taint graph.

Karan Chandra

Judge

10+ years building production fraud, risk, and anomaly detection ML across hundreds of millions of transactions. End-to-end owner: problem framing, feature engineering, deployment, monitoring. I care about models that hold up under real-world scale, not leaderboard scores.

Naman Ahuja

Judge

I am a Software Engineer at Meta and my work includes building AI production safeguards and large-scale infrastructure, alongside research in adversarial evaluation of AI systems.

Goutham Nekkalapu

Judge

Goutham Nekkalapu is a Principal Research Engineer who builds AI-powered products used by millions, including AI agents, Retrieval-Augmented Generation techniques etc. A named inventor on multiple patents, he focuses on turning cutting-edge AI into practical, high-impact products and championing best practices for responsible Generative AI adoption.

Temi Oloyede

Judge

I am a Software Engineer at Box working on AI infrastructure for enterprise-scale retrieval-augmented generation (RAG) systems, where I design and operate platforms supporting document understanding, semantic search, and agentic workflows. Previously, I worked at Facebook (Meta) contributing to large-scale ranking, retrieval, and ML system infrastructure.

Pon Murugesh Devendren

Judge

Pon Murugesh Devendren is an Enterprise AI Architect specializing in SAP BTP and AI governance, designing secure, scalable AI systems for high-stakes infrastructure and decision-making environments.

Abhishek Das

Judge

Abhishek Das is a Staff ML and Distributed Systems Engineer at Salesforce AI Platform, where he leads large-scale DAG-based and agentic inference platforms powering enterprise AI workloads. He has scaled multi-tenant ML inference systems to tens of millions of production requests per day and has deep expertise across distributed systems, ML platforms, cloud architecture, and AI infrastructure. His background includes leadership roles at Salesforce and Microsoft Azure, multiple patents, and extensive experience mentoring engineers and evaluating complex technical systems.

Ashita Khetan

Judge

Ashita Khetan is a Principal Software Engineer at Microsoft with over 12 years of experience building large-scale enterprise and AI-powered productivity solutions used by hundreds of millions of users. She specializes in customer experience technologies, artificial intelligence, cloud platforms, and enterprise software, and contributes to the broader technology community through judging, peer review, and mentoring initiatives.

Sachin Singh

Judge

Sachidananda Singh is a Senior Engineering Manager at Hims & Hers Health, where he leads supply chain engineering. Over seventeen years he has built and led engineering teams across Amazon Web Services, Wayfair, and Windstream, spanning core cloud infrastructure, large-scale logistics platforms, and applied LLM systems in production. At AWS he was a tech lead for the time-synchronization infrastructure behind EC2. His work centers on complex distributed systems where failures are subtle, silent, and expensive to find.

Roshan Choudhary

Judge

Roshan Choudhary is a Lead Engineer and published technical author specializing in Azure cloud-native development, including serverless architecture, Azure Functions, and AI-augmented workflows, with 14+ years of experience delivering solutions across airlines, finance, and insurance. Based in the Research Triangle Park area of North Carolina, he holds a GitHub Copilot certification (GH-300), OCJP, and OCWCD and authored Azure Functions: From Fundamentals to Production, available free on Google Play.

Jananan Mithrakumar

Judge

Jana Mithrakumar is a Stanford EE alum, former AI accelerator chip designer, and YC founder currently working in consumer AI. His work focuses on building and scaling production AI products, with his latest app recently crossing 250,000 users and over $1M in ARR.

Janushi Shastri

Judge

Janushi Shastri is an AI/ML Engineer and AI Lead at NSV Inc., specializing in medical AI with production experience building clinical AI systems for cervical cancer screening and oral cancer detection deployed across Kenya, Zimbabwe, UK, and India, including collaboration with WHO/IARC experts. She holds an IEEE publication in medical imaging AI and has expertise in deep learning, computer vision, model validation, uncertainty estimation, and responsible AI.

Dheeraj Reddy Pailla

Judge

Dheeraj Reddy Pailla is a published researcher in multimodal deep learning and AI systems (ACM Web Science 2021; IEEE/CVF ICCV 2019; AAAI ICWSM 2020; 1,729 citations) and Senior Software Engineer at Sam's Club / Walmart Global Technology building AI infrastructure for tens of millions of U.S. members.

Anchit Jhingan

Judge

Anchit Jhingan is a Senior Data Scientist with over 6 years of experience in big tech, specializing in machine learning, AI systems, and analytics. He currently works at Amazon Prime Video in the content localization domain. His work focuses on building intelligent systems to solve real-world business problems, particularly in media and entertainment.

Jason Dsouza

Judge

Jason Dsouza has been a serial hackathon winner (often the youngest) at some of the largest crypto and AI hackathons (ETHGlobal, Consenys Web3athon, NASA JSC Hackathon, Near, Filecoin etc) b/w 2021-24. He has also judged at some of the larger hackathons in Canada - UofTHacks (700+ participants), Kinghacks, GenAI Genesis (600+ participants) and looking to expand that to other cities and venues.

Vashishtha Patil

Judge

Vashishtha Patil is a Senior Applied Scientist at Amazon, developing the next generation of LLM-powered AI capabilities for the Alexa+ Smart Home experience. With 13 years in machine learning spanning Amazon and Qualcomm, he specializes in bringing AI from research to real-world products.

Arun Aalla

Judge

Arun Aalla is an Applied AI Architect specializing in enterprise AI solutions, with expertise in large language models, agentic AI, and intelligent automation. He partners with organizations to design and deliver scalable, production-ready AI systems that solve complex business challenges and create measurable impact.

Ankit Pandey

Judge

Ankit Pandey is a sales executive focused on driving business transformation and growth through Cloud, Data & AI. With more than a decade of enterprise sales and client-partner experience (new-logo acquisition, account strategy, and GTM partnerships), he has helped C-suite and senior IT leaders convert digital ambition into measurable outcomes across BFSI, retail/e-commerce, healthcare and more. He has an MBA from GWU with advanced credentials in data analytics and practical certifications in cloud and enterprise tools.

Registered Local Sites

Register A Location

Beside the remote and virtual participation, our amazing organizers also host local hackathon locations where you can meet up in-person and connect with others in your area.

The in-person events for the Apart Sprints are run by passionate individuals just like you! We organize the schedule, speakers, and starter templates, and you can focus on engaging your local research, student, and engineering community.

Our Other Sprints

-

Research

AI Incident Response Sprint

This unique event brings together diverse perspectives to tackle crucial challenges in AI alignment, governance, and safety. Work alongside leading experts, develop innovative solutions, and help shape the future of responsible

Sign Up

-

Research

Digital Minds Research Sprint

This unique event brings together diverse perspectives to tackle crucial challenges in AI alignment, governance, and safety. Work alongside leading experts, develop innovative solutions, and help shape the future of responsible

Sign Up

Apart Research logo

Sign up to stay updated on the
latest news, research, and events

Careers

AI Safety Ideas

Code of Conduct

Responsible disclosure policy

sprints@apartresearch.com

Home

Sprints

Fellowships

Impact

Research

Google Scholar icon

Design by Boyne Creative

Apart Research logo

Sign up to stay updated on the
latest news, research, and events

Careers

AI Safety Ideas

Code of Conduct

Responsible disclosure policy

sprints@apartresearch.com

Home

Sprints

Fellowships

Impact

Research

Google Scholar icon

Design by Boyne Creative

Apart Research logo

Sign up to stay updated on the
latest news, research, and events

Careers

AI Safety Ideas

Code of Conduct

Responsible disclosure policy

sprints@apartresearch.com

Home

Sprints

Fellowships

Impact

Research

Google Scholar icon

Design by Boyne Creative

Apart Research logo

Sign up to stay updated on the
latest news, research, and events

Careers

AI Safety Ideas

Code of Conduct

Responsible disclosure policy

sprints@apartresearch.com

Home

Sprints

Fellowships

Impact

Research

Google Scholar icon

Design by Boyne Creative