Skip to content
Sprint projectJul 26, 2026Rawalpindi, Pakistan

Capability Requirements and Worst Case Harms for Secret Loyalties

Misbah Tariq · Team Excalibur

Submitted to Secret Loyalties Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Capability Requirements and Worst Case Harms for Secret Loyalties

Share

Maps Kwon et al.'s activation-breadth/action-breadth space to the concrete capabilities each corner requires (situational awareness, deception, theory of mind, long-horizon planning), then adds a fifth region the taxonomy misses: emergent, zero-attacker loyalty from skewed RLHF feedback. Argues the broad/broad "coup-enabling" corner is the least credible near-term threat given currently demonstrated capabilities, while the narrow/narrow corner, including its emergent variant, is the most likely to already be occupied. Proposes concrete scaling indicators to watch and defensive measures (evaluator-diversity audits, matched-control audits, held-out sign-off panels) targeted specifically at the emergent pathway that existing attacker-centric defenses don't cover.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. This is a polished and thoughtful capability-based threat analysis, particularly in its distinction between currently demonstrated narrow loyalties and more speculative broad, autonomous variants. The most important revision would be to clarify the status of the proposed “zero-attacker emergent loyalty” pathway. The cited RLHF evidence demonstrates unintended sophistry: greater persuasiveness without greater correctness but does not establish undisclosed preference toward a particular evaluator group or identifiable principal. The argument would be stronger either by treating this as an adjacent alignment failure rather than a secret loyalty, or by providing evidence that skewed feedback produces principal-specific favoritism. The proposed warning indicators should also distinguish proxy capabilities from necessary or sufficient conditions, justify their thresholds, and account explicitly for both likelihood and harm when recommending defensive priorities. A visual capability map and clearer labels separating evidence from speculation would further improve the presentation.

    Read full reviewShow less
  2. The mapping is good. It directly answers the track prompts. However the " loyalty" extension is not really a new idea. It is mostly a new name for things that people already know about from RLHF sycophancy findings. To make it better we need to see some examples or experiments. For instance we could do a test to see how people react when the people evaluating them are not very fair. This would make the idea that it's credible in the near term more believable.

    The argument for prioritization makes sense. It is still too general. We need to make the recommendations for defense more specific. This means we should say what labs can do tomorrow like what numbers they should look for how they should keep track of things or what they should check for.

    Overall this is still work that is clear and answers the question. It is the kind of thing you could do over a weekend. But it does not have any new or interesting ideas or any evidence to back it up so it is not the best it could be. The mapping and the ideas, about " loyalty" and prioritization are good but they need more work to be really great.

    Read full reviewShow less
  3. A solid effort. You definitely get points for extending a framework into new domains, but I do not see the case for AI safety here. This reads as a bias paper, and not an AI safety paper. I agree that it is a novel contribution in that it extends the existing framework, but it misses the impact for me. Making the specific threat of this threat model more clear will have helped. The text is written clearly, but feels very AI edited, and could have been significantly shorter or more dense with ideas. A diagram and a summary would have been helpful to clarity the so-what here more.

Cite this project

@misc{tariq2026capability,
  title = {{Capability Requirements and Worst Case Harms for Secret Loyalties}},
  author = {Misbah Tariq},
  year = {2026},
  month = jul,
  note = {Submitted to Secret Loyalties Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/capability-requirements-and-worst-case-harms-for-secret-loyalties-fvta}},
  url = {https://apartresearch.com/sprints/projects/capability-requirements-and-worst-case-harms-for-secret-loyalties-fvta}
}

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026