# Apart Research > Apart Research is a nonprofit AI safety organization. We research and evaluate harmful manipulation and other risks in frontier AI systems, and we run Sprints (weekend research hackathons) and Fellowships (mentored research towards publication) that bring new researchers into AI safety. Website: https://apartresearch.com. Contact: contact@apartresearch.com (general and media), sprints@apartresearch.com (Sprint prizes). Last updated: October 22, 2026. Key facts: - Nonprofit, remote-first and globally distributed. CEO: Jaime Raldúa Veuthey. - Research focus: harmful manipulation by advanced AI, one of the systemic risks under the EU AI Act. Our work supports the EU AI Office, including a harmful-manipulation evaluation tender, the Code of Practice, the AI Act Advisory Forum and the Expert Forum on Frontier AI. - Track record: 60+ publications in AI safety (NeurIPS, ICLR, ICML, AAAI, ACL, Science), 100+ researchers in the Apart Fellowship, 8,700+ Sprint participants, 1,600+ Sprint projects, 50+ Sprints since 2022, participants from 120+ nationalities. - We work with frontier AI labs (targeted model-behavior evaluations, independent testing of safeguards), policymakers and public institutions (empirical studies and technical evidence for policy and enforcement), and AI safety researchers (methods, benchmarks, collaborations). How our programs connect: - Apart Fellowship (anyone can start): Sprint → top 5–20% of teams invited to Apart Studio (about 6–8 weeks) → about 40% of Studio teams selected for the Apart Fellowship (6–16 weeks) → published research. No application form. - Partnered Fellowships (apply directly, no Sprint needed): application → selected by the partner's mentors → about 16 weeks on the partner's research agenda → published research. - Both are remote and part-time (minimum 10 hours per week), with compute, travel and publication support. No stipend by default. ## Work with Apart - [Contact](https://apartresearch.com/contact): Evaluations or testing for an AI provider, policy and public-sector research, research collaborations, Sprints and Fellowships, careers, sponsoring a Sprint or Fellowship, and general inquiries. Please do not send confidential model information, credentials or restricted material through the form; we set up a suitable channel for sensitive discussions. - Funding and donations: Apart is funded by grants and donations. To donate, fund research or sponsor a Sprint or Fellowship, use the contact page (choose "Sponsoring a Sprint or Fellowship" or "General inquiry") or email contact@apartresearch.com. - [Careers](https://apartresearch.com/careers): Open roles and open applications. Currently hiring: [Chief Operating Officer](https://apartresearch.com/careers/chief-operating-officer) (full time, remote (4+ hours overlap with CET working hours preferred)); [Director of Research Programs](https://apartresearch.com/careers/director-of-research-programs) (full-time, remote (4+ hours overlap with CET working hours preferred)); [Special Projects Manager](https://apartresearch.com/careers/special-projects-manager) (full-time or Part-time, remote (4+ hours overlap with CET working hours preferred)); [Fellowships Lead](https://apartresearch.com/careers/fellowships-lead) (full-time, remote (significant working-hour overlap with UTC-6/7 and UTC+1/2)); [AI Cyber Threat Researcher (OSINT)](https://apartresearch.com/careers/ai-cyber-threat-researcher-osint) (full-time or Part-time, remote (4+ hours overlap with CET working hours preferred)); [Research Scientist, Manipulation Evals](https://apartresearch.com/careers/research-scientist-manipulation-evals) (full time, remote, Europe or San Francisco preferred). - Mentors and partners: Mentor Fellowship teams, or run or sponsor a Partnered Fellowship on your research agenda. Apart handles recruiting, project management and compute. [Apply as a mentor](https://apartresearch.notion.site/2b8fcfd1de9d80979154c372abd51522) or [apply as a partner](https://apartresearch.notion.site/240fcfd1de9d80839b5beb781fe4482d). - [Propose a partner hackathon](https://apartresearch.com/sprints/collaborations/submit): Apart can host your AI safety hackathon's page, sign-ups and project submissions. ## Research - [Research](https://apartresearch.com/research): Apart's publications and benchmarks, with filters by research area (manipulation, deception, model evaluation, AI governance, dangerous capabilities, AI agents). - [All publications on Google Scholar](https://scholar.google.com/citations?hl=en&user=f2-xhQkAAAAJ): Apart Research's full publication list. - [The science and practice of proportionality in AI risk evaluations](https://doi.org/10.1126/science.aea3835): Science 2026, Policy Forum. AI evaluations should provide meaningful risk information without imposing excessive burden. - [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://proceedings.neurips.cc/paper_files/paper/2025/hash/aa95d031f5e151bf8fb402f44bd97054-Abstract-Conference.html): NeurIPS 2025. Detects hidden capabilities by testing how models respond to noise in their weights. - [DarkBench: Benchmarking Dark Patterns in Large Language Models](https://arxiv.org/abs/2503.10728): ICLR 2025, Oral. A benchmark of 660 prompts for manipulation, sycophancy and other dark patterns in leading language models. Benchmark site: https://darkbench.ai/ - [Multi-Agent Risks from Advanced AI](https://arxiv.org/abs/2502.14143): Cooperative AI Foundation 2025, technical report. Maps the failure modes that emerge when many AI agents interact. - [Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systems](https://openreview.net/forum?id=ljBR8i3jmX): AAAI 2025. - [Rethinking CyberSecEval: An LLM-Aided Approach to Evaluation Critique](https://arxiv.org/abs/2411.08813): NeurIPS 2024 workshop. - [CryptoFormalEval: Integrating LLMs and Formal Verification for Automated Cryptographic Protocol Vulnerability Detection](https://openreview.net/forum?id=hqvJT2pOxu): NeurIPS 2024 workshop. - [Sandbag Detection through Model Impairment](https://openreview.net/forum?id=zpcx5a8w2a): NeurIPS 2024 workshop. - [Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities](https://arxiv.org/abs/2410.09114): arXiv preprint 2024. - [Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts](https://arxiv.org/abs/2410.09247): ICML 2024 workshop. - [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://arxiv.org/abs/2401.05566): arXiv preprint 2024. An Anthropic-led paper; Apart contributed as a collaborator. - [Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark](https://aclanthology.org/2023.findings-acl.733/): Findings of ACL 2023. - [Responsible Disclosure Policy](https://apartresearch.com/info/responsible-disclosure-policy): How Apart reviews and limits publication of dual-use research, including Sprint projects. ## Programs - [Programs](https://apartresearch.com/programs): Overview of Sprints and Fellowships and how they connect. - [Sprints](https://apartresearch.com/sprints): Weekend research hackathons on one AI safety question, about once a month, online and at local sites. Teams build an experiment, evaluation, dataset or tool. No AI safety background needed; teams form in Discord at the start. - [All Sprints](https://apartresearch.com/sprints/all): Every Apart Sprint since 2022, with dates, format and submitted projects. - [Sprint projects](https://apartresearch.com/sprints/projects): Search the 1,600+ projects teams have submitted. - [Local Sprint sites](https://apartresearch.com/sprints/locations): Community groups that host in-person sites for Sprints. - [Partner hackathons](https://apartresearch.com/sprints/collaborations): Hackathons run by universities and partner organizations with Apart's support. - [Sprint prize terms](https://apartresearch.com/sprints/prize-terms): Prizes are paid by bank transfer through Apart's fiscal sponsor, the registered 501(c)(3) Ashgro Inc.; team prizes are split equally. - [Fellowships](https://apartresearch.com/fellowships): The Apart Fellowship (by invitation after a Sprint) and Partnered Fellowships (apply directly), with FAQs on time commitment, support, pay and authorship. - [Apart Fellowship](https://apartresearch.com/fellowships/apart-fellowship): Stand out at a Sprint, develop the idea in Apart Studio, then 6–16 weeks of research with a research project manager, compute and publication support. - [Partnered Fellowships](https://apartresearch.com/fellowships/partnered-fellowships): Apply to research projects set by partner organizations, who set the agenda and mentor; Apart runs the program. Former program names: "Apart Lab" and "Apart Lab Fellowship" are now the Apart Fellowship; "Apart Lab Studio" is now Apart Studio; "Alignment Jams" and "hackathons" are now Sprints; "Jam sites" are now local Sprint sites. ## Sprints Upcoming and past Apart Sprints, newest first. Each page lists the brief, tracks, format and submitted projects. - [Threat Intelligence & Threat Modelling Hackathon](https://apartresearch.com/sprints/threat-intelligence-threat-modelling-2026): Upcoming, Dec 4–6, 2026. Can LLMs scale up threat intelligence and threat modelling for AI data centres facing nation-state attackers? With Security Level 5 (SL5). - [Agent Attribution Hackathon](https://apartresearch.com/sprints/agent-attribution-2026): Upcoming, Nov 27–29, 2026. Telling rogue AI agents apart from human-steered ones. One weekend to build attribution classifiers, simulate agents that evade them, and find rogue agents in the wild. - [AI x Epistemics Research Sprint](https://apartresearch.com/sprints/ai-x-epistemics-research-sprint-2026-11-13-to-2026-11-15): Upcoming, Nov 13–15, 2026. AI for epistemics: whether models know how solid their claims are, trust and provenance infrastructure, and products that improve decisions. Online. - [Swarm Alignment Hackathon](https://apartresearch.com/sprints/swarm-alignment-2026): Upcoming, Nov 6–8, 2026. Alignment can't stop at single models when AI agents start to negotiate, trade and share tools. Spend a weekend uncovering and preventing failures that emerge between multiple agents. - [AI Collusion Research Sprint](https://apartresearch.com/sprints/ai-collusion-research-sprint-2026-10-23-to-2026-10-25): Upcoming, Oct 23–25, 2026. Collusion between AI agents in markets and workflows: detecting, auditing and breaking it. Co-organized with Poseidon Research and AE Studio, supported by Modal and the AI Alignment Foundation; online with hubs at Collider (New York City) and AI Safety Hong Kong. - [AI Incident Response Sprint](https://apartresearch.com/sprints/ai-incident-response-sprint-2026-09-11-to-2026-09-13): Past, Sep 11–13, 2026. Incident response for autonomous AI agents, using the public record of the July 2026 OpenAI–Hugging Face agent intrusion. - [Digital Minds Research Sprint](https://apartresearch.com/sprints/digital-minds-research-sprint-2026-08-14-to-2026-08-16): Past, Aug 14–16, 2026. Preferences, welfare signals, introspection and identity in frontier AI models. - [Secret Loyalties Hackathon](https://apartresearch.com/sprints/secret-loyalties-hackathon-2026-07-24-to-2026-07-26): Past, Jul 24–26, 2026. Secretly loyal AI: models that covertly advance a hidden principal's interests while appearing normal. - [Global South AI Safety Hackathon](https://apartresearch.com/sprints/global-south-ais-hackathon-2026-06-19-to-2026-06-21): Past, Jun 19–21, 2026. First AI safety hackathon designed for and with the Global South, across 7 regional hubs in Latin America, Africa and Asia. - [The Secure Program Synthesis Hackathon](https://apartresearch.com/sprints/secure-program-synthesis-hackathon-2026-05-22-to-2026-05-24): Past, May 22–24, 2026. AI, formal methods and security: tools that catch what AI code generation gets wrong. Co-organized with Atlas Computing. - [AIxBio Hackathon](https://apartresearch.com/sprints/aixbio-hackathon-2026-04-24-to-2026-04-26): Past, Apr 24–26, 2026. Biosecurity tools at the intersection of AI and biology, such as DNA synthesis screening and pandemic early warning. - [AI Control Hackathon 2026](https://apartresearch.com/sprints/ai-control-hackathon-2026-03-20-to-2026-03-22): Past, Mar 20–22, 2026. AI control: techniques that keep untrusted AI systems safe to deploy, including monitoring and sabotage detection. - [The Technical AI Governance Challenge](https://apartresearch.com/sprints/the-technical-ai-governance-challenge-2026-01-30-to-2026-02-01): Past, Jan 30–Feb 1, 2026. Verification systems and technical infrastructure for international AI governance. - [AI Manipulation Hackathon](https://apartresearch.com/sprints/ai-manipulation-hackathon-2026-01-09-to-2026-01-11): Past, Jan 9–11, 2026. Measurement and defense tools against AI manipulation. - [Defensive Acceleration Hackathon](https://apartresearch.com/sprints/def-acc-hackathon-2025-11-21-to-2025-11-23): Past, Nov 21–23, 2025. Societal defenses against AI-enabled threats. - [The AI Forecasting Hackathon](https://apartresearch.com/sprints/the-ai-forecasting-hackathon-2025-10-31-to-2025-11-02): Past, Oct 31–Nov 2, 2025. Forecasting AI progress and its impacts. - [ARENA 6.0 Mechanistic Interpretability Hackathon](https://apartresearch.com/sprints/arena-6-mechanistic-interpretability-hackathon-2025-09-13-to-2025-09-14): Past, Sep 13–14, 2025. Mechanistic interpretability projects with the ARENA 6.0 cohort. - [CBRN AI Risks Research Sprint](https://apartresearch.com/sprints/cbrn-ai-risks-sprint-2025-09-12-to-2025-09-14): Past, Sep 12–14, 2025. Chemical, biological, radiological and nuclear risks from AI. - [AI Safety x Physics Grand Challenge](https://apartresearch.com/sprints/ai-safety-x-physics-grand-challenge-2025-07-25-to-2025-07-27): Past, Jul 25–27, 2025. Applying physics methods to AI safety problems. - [Red Teaming A Narrow Path: ControlAI Policy Sprint](https://apartresearch.com/sprints/red-teaming-a-narrow-path-control-ai-policy-sprint-2025-06-13-to-2025-06-13): Past, Jun 13, 2025. Stress-testing ControlAI's "A Narrow Path" policy proposals. - [Apart x Martian Mechanistic Router Interpretability Hackathon](https://apartresearch.com/sprints/apart-x-martian-mechanistic-router-interpretability-hackathon-2025-05-30-to-2025-06-01): Past, May 30–Jun 1, 2025. Interpretable judge models and routing for expert orchestration, with Martian. - [Economics of Transformative AI](https://apartresearch.com/sprints/economics-of-transformative-ai-research-sprint-2025-04-25-to-2025-04-27): Past, Apr 25–27, 2025. Economic effects of transformative AI. - [Dark Patterns in AGI Hackathon at ZAIA](https://apartresearch.com/sprints/dark-patterns-in-agi-hackathon-at-zaia-2025-04-04-to-2025-04-05): Past, Apr 4–6, 2025. Dark patterns and manipulation in AI systems, with the ZAIA community. - [AI Control Hackathon 2025](https://apartresearch.com/sprints/ai-control-hackathon-2025-03-29-to-2025-03-30): Past, Mar 29–30, 2025. AI control protocols and settings. - [Women in AI Safety Hackathon](https://apartresearch.com/sprints/women-in-ai-safety-hackathon-2025-03-07-to-2025-03-14): Past, Mar 7–10, 2025. AI safety hackathon highlighting women in the field; Apart's biggest event at the time. - [AI Safety Entrepreneurship Hackathon](https://apartresearch.com/sprints/ai-safety-entrepreneurship-hackathon-2025-01-17-to-2025-01-20): Past, Jan 17–20, 2025. Building AI safety startups. - [Reprogramming AI Models Hackathon](https://apartresearch.com/sprints/reprogramming-ai-models-hackathon-2024-11-22-to-2024-11-25): Past, Nov 22–25, 2024. Interpretability and feature steering, with Goodfire's API. - [AI Policy Hackathon at Johns Hopkins University](https://apartresearch.com/sprints/ai-policy-hackathon-at-johns-hopkins-university-2024-10-26-to-2024-10-28): Past, Oct 26–28, 2024. AI policy proposals, hosted at Johns Hopkins University. - [Agent Security Hackathon](https://apartresearch.com/sprints/agent-security-hackathon-2024-10-04-to-2024-10-07): Past, Oct 4–7, 2024. Security of LLM agents. - [ARENA 4.0 Interpretability Hackathon](https://apartresearch.com/sprints/arena-4-interp-hackathon-2024-09-15-to-2024-09-15): Past, Sep 15, 2024. Interpretability projects with the ARENA 4.0 cohort. - [The Concordia Contest](https://apartresearch.com/sprints/the-concordia-contest-2024-09-06-to-2024-09-09): Past, Sep 6–9, 2024. Cooperative intelligence of language model agents. - [Hackathon for Technical AI Safety Startups](https://apartresearch.com/sprints/hackathon-for-technical-ai-safety-startups-2024-09-22-to-2024-09-25): Past, Aug 30–Sep 2, 2024. Technical AI safety startup ideas. - [AI capabilities and risks demo-jam](https://apartresearch.com/sprints/ai-capabilities-and-risks-demo-jam-2024-08-23-to-2024-08-26): Past, Aug 23–26, 2024. Interactive demonstrations of AI capabilities and risks. - [Research Augmentation Hackathon](https://apartresearch.com/sprints/research-augmentation-hackathon-2024-07-26-to-2024-07-29): Past, Jul 26–29, 2024. Tools that speed up AI alignment research. - [Deception Detection Hackathon: Preventing AI deception](https://apartresearch.com/sprints/deception-detection-hackathon-preventing-ai-deception-2024-06-28-to-2024-07-01): Past, Jun 28–Jul 1, 2024. Detecting deception in AI systems, with Apollo Research. - [Computational Mechanics Hackathon](https://apartresearch.com/sprints/computational-mechanics-hackathon-2024-06-01-to-2024-06-03): Past, Jun 1–3, 2024. Interpretability through computational mechanics. - [AI Security Evaluation Hackathon: Measuring AI Capability](https://apartresearch.com/sprints/ai-security-evaluation-hackathon-2024-05-24-to-2024-05-27): Past, May 24–27, 2024. Benchmarks that measure dangerous AI capabilities. - [AI and Democracy Hackathon: Demonstrating the Risks](https://apartresearch.com/sprints/ai-and-democracy-hackathon-2024-05-03-to-2024-05-06): Past, May 3–6, 2024. Demonstrating and mitigating AI risks to democracy. - [InterpVis](https://apartresearch.com/sprints/interpvis-2024-03-12-to-2024-03-14): Past, Apr 13–14, 2024. Visualizations for mechanistic interpretability, at the LISA offices in London. - [LLM Psychology Hackathon](https://apartresearch.com/sprints/llm-psychology-hackathon-2024-04-05-to-2024-04-07): Past, Apr 5–7, 2024. Studying LLM behavior with methods from psychology. - [Code Red Hackathon: Can LLMs Replicate and do R&D by Themselves?](https://apartresearch.com/sprints/code-red-hackathon-2024-03-22-to-2024-03-25): Past, Mar 22–25, 2024. Evaluation tasks for autonomous replication and AI R&D, with METR. - [Multi-Agent Security Research Sprint](https://apartresearch.com/sprints/multi-agent-security-research-sprint-2024-02-09-to-2024-02-11): Past, Feb 9–11, 2024. Securing multi-agent systems in deployment. - [The AI Governance Research Sprint](https://apartresearch.com/sprints/the-ai-governance-research-sprint-2024-01-05-to-2024-05-07): Past, Jan 5–7, 2024. AI governance research. - [AI Model Evaluations Hackathon](https://apartresearch.com/sprints/ai-model-evaluations-hackathon-2023-11-24-to-2023-11-26): Past, Nov 24–26, 2023. Evaluations of AI model behavior. - [Multi-Agent Safety Hackathon](https://apartresearch.com/sprints/multi-agent-safety-hackathon-2024-09-29-to-2023-11-01): Past, Sep 29–Oct 1, 2023. Multi-agent safety, with the Cooperative AI Foundation. - [The Agency Foundations Challenge](https://apartresearch.com/sprints/the-agency-foundations-challenge-2023-09-08-to-2023-09-24): Past, Sep 8–24, 2023. Safeguarding human agency against advanced AI. - [Distillation Write-a-thon 2.0](https://apartresearch.com/sprints/distillation-write-a-thon-2023-08-24-to-2023-08-26): Past, Aug 24–26, 2023. Writing accessible explanations of AI safety research. - [LLM Evals Hackathon](https://apartresearch.com/sprints/llm-evals-hackathon-2023-08-18-to-2023-08-20): Past, Aug 18–20, 2023. Evaluating risks of deployed language models. - [Interpretability Hackathon 3.0](https://apartresearch.com/sprints/interpretability-hackathon-2023-07-14-to-2023-07-16): Past, Jul 14–16, 2023. Interpretability of language models. - [Safety Benchmarks Hackathon](https://apartresearch.com/sprints/safety-benchmarks-hackathon-2023-06-30-to-2023-07-02): Past, Jun 30–Jul 2, 2023. Safety benchmarks for large models. - [Distillation Write-a-thon](https://apartresearch.com/sprints/ai-governance-hackathon-2023-06-16-to-2023-06-18): Past, Jun 16–18, 2023. Writing accessible explanations of AI safety research. - [ML Verifiability Hackathon](https://apartresearch.com/sprints/ml-verifiability-hackathon-2023-05-26-to-2023-05-28): Past, May 26–28, 2023. Formal and informal verification of ML systems. - [The Interpretability Hackathon 2.0](https://apartresearch.com/sprints/the-interpretability-hackathon-2023-04-14-to-2023-04-17): Past, Apr 14–17, 2023. Mechanistic interpretability. - [AI Governance](https://apartresearch.com/sprints/ai-governance-2023-03-24-to-2023-03-27): Past, Mar 24–27, 2023. AI governance and society. - [Scale Oversight for Machine Learning Hackathon](https://apartresearch.com/sprints/scale-oversight-for-machine-learning-hackathon-2023-02-10-to-2023-02-13): Past, Feb 10–13, 2023. Measuring and monitoring the safety of large ML models. - [Mechanistic Interpretability Hackathon](https://apartresearch.com/sprints/mechanistic-interpretability-hackathon-2023-01-20-to-2023-01-23): Past, Jan 20–23, 2023. Mechanistic interpretability. - [EAGx LatAm Epoch AI Hackathon](https://apartresearch.com/sprints/eagx-latam-epoch-ai-hackathon-2023-01-09-to-2023-01-11): Past, Jan 9–11, 2023. Trends in AI development, with Epoch AI, after EAGx LatAm. - [AI Testing](https://apartresearch.com/sprints/ai-testing-2022-12-15-to-2022-12-17): Past, Dec 15–17, 2022. Testing AI systems. - [The Interpretability Hackathon](https://apartresearch.com/sprints/the-interpretability-hackathon-2022-11-11-to-2022-11-13): Past, Nov 11–13, 2022. Interpretability. - [Language Model Hackathon](https://apartresearch.com/sprints/language-model-hackathon-2022-09-29-to-2022-10-01): Past, Sep 29–Oct 1, 2022. Language model behavior (the first Alignment Jam). ## Partnered Fellowships No Partnered Fellowship is taking applications right now. - [The Heron AI Security Fellowship](https://apartresearch.com/fellowships/the-ai-security-fellowship): Finished, January–April 2026. With Heron AI Security; cybersecurity professionals working with AI security field leaders. - [The Secure Program Synthesis Fellowship](https://apartresearch.com/fellowships/the-secure-program-synthesis-fellowship): Finished, June–September 2026. Mentor-led projects on formal methods, AI systems and security. - [The Martian Fellowship on Mechanistic Interpretability](https://apartresearch.com/fellowships/the-martian-fellowship-on-mechanistic-interpretability): Finished, September–December 2025. With Martian; over $70,000 in research support for fellows and two teams accepted to NeurIPS 2025 workshops. ## Partner hackathons - [Berkeley AI Policy Hackathon](https://apartresearch.com/sprints/collaborations/berkeley-ai-policy-hackathon-2025-04-14-to-2025-04-26): Apr 14–26, 2025. Student policy proposals on AI, in California and beyond. - [Georgia Tech AISI Policy Hackathon](https://apartresearch.com/sprints/collaborations/georgia-tech-aisi-policy-hackathon-2025-04-05-to-2025-04-06): Apr 5–6, 2025. AI governance hackathon organized by Georgia Tech. - [Autostructures: interfaces not between humans and AI, but between humans via AI](https://apartresearch.com/sprints/collaborations/blackpoolautostructuring-2024-11-23-to-2024-11-25): Nov 23–25, 2024. Design for infrastructure embedded with AI. - [Howard University AI Safety Summit & Policy Hackathon](https://apartresearch.com/sprints/collaborations/howard-university-ai-safety-summit-policy-hackathon-2024-11-21-to-2024-11-22): Nov 21–22, 2024. AI governance and safety policy, organized by Howard University. ## Local Sprint sites - [AI Alignment Bangalore](https://apartresearch.com/sprints/locations/ai-alignment-bangalore): Bangalore, India - [AI Safety Arc](https://apartresearch.com/sprints/locations/ai-safety-arc): NIT Agartala, India - [AI safety initiative at Georgia Tech](https://apartresearch.com/sprints/locations/ai-safety-initiative-at-georgia-tech): Atlanta, GA, United States - [CEEALAR](https://apartresearch.com/sprints/locations/ceealar): Blackpool, United Kingdom - [Effective Altruism Denmark](https://apartresearch.com/sprints/locations/effective-altruism-denmark): Copenhagen, Denmark - [Effective Altruism Singapore](https://apartresearch.com/sprints/locations/effective-altruism-singapore): Singapore - [Singapore AI Safety Hub](https://apartresearch.com/sprints/locations/singapore-ai-safety-hub): Singapore - [Technology and Policy Society at Johns Hopkins](https://apartresearch.com/sprints/locations/technology-and-policy-society-at-johns-hopkins): Baltimore, Maryland, United States ## Organization - [Home](https://apartresearch.com/): Apart's research on AI manipulation, programs, and how to work with us. - [About](https://apartresearch.com/about): Mission, how we work, impact numbers, team, board and advisors. - [News](https://apartresearch.com/news): Research releases, Sprint round-ups, researcher spotlights and the newsletter archive. - [Media kit](https://apartresearch.com/media-kit): Boilerplate, background and event photos for journalists. Media inquiries: contact@apartresearch.com. - [Discord](https://discord.gg/dYUWDm7Ben): Community for forming Sprint teams, mentor Q&A and discussing research. - Elsewhere: [GitHub](https://github.com/apartresearch), [X](https://x.com/apartresearch), [LinkedIn](https://www.linkedin.com/company/apartresearch/), [YouTube](https://www.youtube.com/channel/UCnfBOJnTkE9sgjMOOsQbi2w). ## Selected news - [Explaining the Apart Research Fellowships](https://apartresearch.com/news/explaining-the-apart-research-fellowships): Dec 3, 2025. How the Apart Fellowship works, and the launch of Partnered Fellowships. - [Problem Areas in Physics and AI Safety](https://apartresearch.com/news/problem-areas-in-physics-and-ai-safety): Jul 25, 2025. Five problem areas where physics methods can help AI safety. - [Beyond Monolithic AI: The Case for an Expert Orchestration Architecture](https://apartresearch.com/news/beyond-monolithic-ai-the-case-for-an-expert-orchestration-architecture): May 14, 2025. Routing queries across many specialized models instead of one monolithic model. - [Engineering a World Designed for Safe Superintelligence](https://apartresearch.com/news/engineering-a-world-designed-for-safe-superintelligence): Apr 9, 2025. Esben Kran's talk at the Paris AI Action Summit. - [Women in AI Safety: Hackathon Round-Up](https://apartresearch.com/news/women-in-ai-safety-hackathon-round-up): Apr 3, 2025. Winners of Apart's largest Sprint at the time. - [Uncovering Model Manipulation with DarkBench](https://apartresearch.com/news/uncovering-model-manipulation-with-darkbench): Feb 18, 2025. The DarkBench benchmark for manipulative dark patterns in popular LLMs. - [Studio Progress Report](https://apartresearch.com/news/studio-progress-report): Feb 13, 2025. Results from the first Apart Studio cohort. - [AI Hackers in the Wild: LLM Agent Honeypot](https://apartresearch.com/news/ai-hackers-in-the-wild-llm-agent-honeypot): Dec 31, 2024. A honeypot for detecting LLM-based attackers in the wild. - [Apart in 2024](https://apartresearch.com/news/apart-in-2024): Jan 1, 2025. Review of Apart's 2024. - [‘3cb’: The Catastrophic Cyber Capabilities Benchmark](https://apartresearch.com/news/3cb-the-catastrophic-cyber-capabilities-benchmark): Oct 30, 2024. A benchmark for LLM agent cyber-offense capabilities. - [Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts](https://apartresearch.com/news/benchmark-inflation-revealing-llm-performance-gaps-using-retro-holdouts): Oct 18, 2024. Public benchmarks can overstate performance because test data leaks into training data. - [The ultimate guide to AI safety research hackathons](https://apartresearch.com/news/the-ultimate-guide-to-ai-safety-research-hackathons): May 17, 2024. How to get the most out of a research Sprint. - [How to organize a research hackathon](https://apartresearch.com/news/how-to-organize-a-research-hackathon): Mar 1, 2024. Guide for local organizers. ## Policies - [Responsible Disclosure Policy](https://apartresearch.com/info/responsible-disclosure-policy): How Apart handles dual-use research and info hazards. - [Code of Conduct](https://apartresearch.com/info/code-of-conduct): Expected conduct at Apart events and in the community. - [Privacy Policy](https://apartresearch.com/info/privacy-policy): How Apart collects, uses and protects personal data from program participants. ## Optional - [Full site content](https://apartresearch.com/llms-full.txt): The text of Apart's main pages (about, research, programs, fellowships, FAQs and terms) in one file. - [Every Sprint project](https://apartresearch.com/sprints/projects/llms.txt): Index of all 1,630 submitted Sprint projects, grouped by Sprint. Selected award-winning Sprint projects: - [Project Anchored](https://apartresearch.com/sprints/projects/project-anchored-q4fn): 2nd Place Winner, Digital Minds Research Sprint. Anchoring vignettes are the standard survey-methodology fix for self-reports that are not comparable across respondents. This project… - [Readable but Not Causal: Limits of Self-Attributed Welfare Representations in Language Models](https://apartresearch.com/sprints/projects/readable-but-not-causal-limits-of-selfattributed-welfare-representations-in-language-models-ehin): 1st Place Winner, Digital Minds Research Sprint. Welfare-like internal representations are increasingly studied as candidate evidence about AI systems. Their entity attribution—whether a… - [One Dial, Not a Tree: Occupational Personas and Emergent Misalignment](https://apartresearch.com/sprints/projects/one-dial-not-a-tree-occupational-personas-and-emergent-misalignment-ukf6): 4th Place Winner, Digital Minds Research Sprint. Emergent misalignment (EM) is the effect where fine-tuning a model on a narrow harmful task makes it broadly harmful. It is already known… - [Sisyphus in the loop: What Makes an LLM Persist?](https://apartresearch.com/sprints/projects/sisyphus-in-the-loop-what-makes-an-llm-persist-yqd9): 5th Place Winner, Digital Minds Research Sprint. As large language models (LLMs) are increasingly deployed as agents that pursue goals over extended horizons, understanding what determines… - [Model, Instance, or Persona? Measuring Affective Signals in Public Text After an AI Is Retired](https://apartresearch.com/sprints/projects/model-instance-or-persona-measuring-aective-signals-in-public-text-after-an-ai-is-retired-hj7u): 3rd Place Winner, Digital Minds Research Sprint. This sprint asks whether the assistant identifies as a model, an instance, or a persona. I ask which of the three its users name. When a… - [Omission Attacks: When Doing Nothing Is the Attack](https://apartresearch.com/sprints/projects/omission-attacks-when-doing-nothing-is-the-attack-0y1v): 1st Place Winner + LinuxArena Track Winner, AI Control Hackathon 2026. AI control protocols monitor agent actions to detect sabotage, but omission attacks exploit what the agent fails to do rather than what it… - [Syscon: Kernel-Level Syscall Monitoring for AI Agent Control in Sandboxed Environments](https://apartresearch.com/sprints/projects/syscon-kernellevel-syscall-monitoring-for-ai-agent-control-in-sandboxed-environments-guun): LinuxArena Track Winner, AI Control Hackathon 2026. Many existing AI control protocols rely on a trusted monitor to evaluate an untrusted agent's actions, but these monitors generally access… - [Detecting LLM Subversion in Vulnerability Patching Settings](https://apartresearch.com/sprints/projects/detecting-llm-subversion-in-vulnerability-patching-settings-gagj): 2nd Place Winner, AI Control Hackathon 2026. LLMs are increasingly used to propose fixes to vulnerabilities in code. If the LLM is misaligned or untrustworthy, it may propose fixes… - [Undoing the Fix: CVE Regression as an AI Control Setting](https://apartresearch.com/sprints/projects/undoing-the-fix-cve-regression-as-an-ai-control-setting-f855): 4th Place Winner, AI Control Hackathon 2026. Production codebases carry accumulated security fixes that can be lost when an AI agent modifies code, whether deliberately or by mistake.… - [Counterfactual Components for AI Control Protocols](https://apartresearch.com/sprints/projects/counterfactual-components-for-ai-control-protocols-ibli): 5th Place Winner, AI Control Hackathon 2026. Current AI control protocols rely heavily on monitors scoring suspiciousness — a single point of failure vulnerable to prompt injection and… - [ActionLens: Pre-Execution Environment Probing for Agent Action Approval](https://apartresearch.com/sprints/projects/actionlens-preexecution-environment-probing-for-agent-action-approval-xy22): 3rd Place Winner, AI Control Hackathon 2026. ActionLens is a pre-execution control protocol for shell and file actions proposed by AI agents. Instead of approving an action from… - [Markov Chain Lock Watermarking: Provably Secure Authentication for LLM Outputs](https://apartresearch.com/sprints/projects/markov-chain-lock-watermarking-provably-secure-authentication-for-llm-outputs-l8oe): 2nd Place Winner, The Technical AI Governance Challenge. We present Markov Chain Lock (MCL) watermarking, a cryptographically secure framework for authenticating LLM outputs. MCL constrains token… - [Prototyping an Embedded Off-Switch for AI Compute](https://apartresearch.com/sprints/projects/prototyping-an-embedded-offswitch-for-ai-compute-65pz): 5th Place Winner, The Technical AI Governance Challenge. This project prototypes an embedded off-switch for AI accelerators. The security block requires periodic cryptographic authorization to… - [Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors](https://apartresearch.com/sprints/projects/fingerprinting-all-ai-cluster-io-without-mutually-trusted-processors-d81l): 4th Place Winner, The Technical AI Governance Challenge. We design and simulate a "border patrol" device for generating cryptographic evidence of data traffic entering and leaving an AI cluster,… - [Political Intelligence for AI Safety: The AI Risk Attitudes Survey (AIRAS)](https://apartresearch.com/sprints/projects/political-intelligence-for-ai-safety-the-ai-risk-attitudes-survey-airas-dlpz): 3rd Place Winner, The Technical AI Governance Challenge. The AI safety and governance community is making progress on defining red lines around existential risk from advanced AI systems, and… - [LidaSim: Testing AI Policies With Persona-Based Simulations](https://apartresearch.com/sprints/projects/lidasim-testing-ai-policies-with-personabased-simulations-o8x6): 1st Place Winner, The Technical AI Governance Challenge. We simulate well-known figures in AI and politics with agents, scraping large amounts of data to get realistic simulations. Then, we test… - [Eliciting Deception on Generative Search Engines](https://apartresearch.com/sprints/projects/eliciting-deception-on-generative-search-engines-6nuu): 2nd Place Winner, AI Manipulation Hackathon. Large language models (LLMs) with web browsing capabilities are vulnerable to adversarial content injection—where malicious actors embed… - [SycophantSee - Activation-based diagnostics for prompt engineering: monitoring sycophancy at prompt and generation time](https://apartresearch.com/sprints/projects/sycophantsee-activationbased-diagnostics-for-prompt-engineering-monitoring-sycophancy-at-prompt-and-generation-time-ys27): 5th Place Winner, AI Manipulation Hackathon. Activation monitoring reveals that prompt framing affects a model's internal state before generation begins. - [Who Does Your AI Serve? Manipulation By and Of AI Assistants](https://apartresearch.com/sprints/projects/who-does-your-ai-serve-manipulation-by-and-of-ai-assistants-77xx): 1st Place Winner, AI Manipulation Hackathon. AI assistants can be both instruments and targets of manipulation. In our project, we investigated both directions across three studies. AI… - [Cross-Linguistic Sycophancy in Frontier LLMs: A Benchmark Study](https://apartresearch.com/sprints/projects/crosslinguistic-sycophancy-in-frontier-llms-a-benchmark-study-w55u): 3rd Place Winner, AI Manipulation Hackathon. We developed a cross-linguistic sycophancy benchmark testing whether frontier AI models exhibit different manipulation behaviours across… - [Agent Attacks via Memory Injection](https://apartresearch.com/sprints/projects/agent-attacks-via-memory-injection-izfz): 4th Place Winner, AI Manipulation Hackathon. LLMs have recently adopted persistent memory to provide models with better user knowledge and personalisation. However, this introduces a… - [Local Learning Coefficients Predict Developmental Milestones During Group Relative Policy Optimization](https://apartresearch.com/sprints/projects/local-learning-coefficients-predict-developmental-milestones-during-group-relative-policy-optimization-2te2): Prize winner, AI Safety x Physics Grand Challenge. In this work, we investigate the emergence of capabilities in reinforcement learning (RL) by framing them as developmental phase… - [AI agentic system epidemiology](https://apartresearch.com/sprints/projects/ai-agentic-system-epidemiology-b83r): Prize winner, AI Safety x Physics Grand Challenge. As AI systems scale into decentralized, multi-agent deployments, emergent vulnerabilities challenge our ability to evaluate and manage… - [Momentum–Point-Perplexity Mechanics in Large Language Models=](https://apartresearch.com/sprints/projects/momentumpointperplexity-mechanics-in-large-language-models-v1nc): Prize winner, AI Safety x Physics Grand Challenge. This work analyzes the hidden states of twenty different open-source transformer language models, ranging from small to medium size and… - [Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process](https://apartresearch.com/sprints/projects/constrained-belief-updates-and-geometric-structures-in-transformer-representations-for-the-rrxor-process-gsok): Prize winner, AI Safety x Physics Grand Challenge. In this work, we seek to further understand the computational structure formed by transformers trained to predict the next token of a data… - [Manipulating Self-Preference for Large Language Models](https://apartresearch.com/sprints/projects/manipulating-selfpreference-for-large-language-models-c4wm): 1st place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. Large language models (LLMs) carry great value as evaluators of synthetic data for research and production settings. However, recent… - [Approximating Human Preferences Using a Multi-Judge Learned System](https://apartresearch.com/sprints/projects/approximating-human-preferences-using-a-multijudge-learned-system-v3im): 2nd place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. In this work, we introduced a learned approach to aggregating multi-judge scores: using a GAM and a simple MLP as an alternative to… - [Guardian-Loop: Mechanistically Interpretable Micro-Judges with Adversarial Self-Improvement](https://apartresearch.com/sprints/projects/guardianloop-mechanistically-interpretable-microjudges-with-adversarial-selfimprovement-1b2p): 4th place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. Guardian-Loop is a mechanistically interpretable judge system designed to enhance the Expert Orchestration Architecture through transparent… - [Judge using SAE Features](https://apartresearch.com/sprints/projects/judge-using-sae-features-y1r1): 3rd place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. The key idea of this project was to explore model judgement using Sparse Autoencoder (SAE) features for mathematical reasoning tasks… - [Economic Feasibility of Universal High Income (UHI) in an Age of Advanced Automation](https://apartresearch.com/sprints/projects/economic-feasibility-of-universal-high-income-uhi-in-an-age-of-advanced-automation-jn3g): 4th place by peer review, Economics of Transformative AI. This paper analyzes five interlinked fiscal measures proposed to fund a Universal High Income (UHI) system in response to large-scale… - [Evaluating the risk of job displacement by transformative AI automation in developing countries: A case study on Brazil](https://apartresearch.com/sprints/projects/evaluating-the-risk-of-job-displacement-by-transformative-ai-automation-in-developing-countries-a-case-study-on-brazil-829h): 3rd place by peer review, Economics of Transformative AI. In this paper, we introduce an empirical and reproducible approach to monitoring job displacement by TAI. We first classify occupations… - [The Early Economic Impacts of Transformative AI: A Focus on Temporal Coherence](https://apartresearch.com/sprints/projects/the-early-economic-impacts-of-transformative-ai-a-focus-on-temporal-coherence-ipql): 1st place by peer review, Economics of Transformative AI. We investigate the economic potential of Transformative AI, focusing on "temporal coherence"—the ability to maintain goal-directed behavior… - [Economics of AI Data Center Energy Infrastructure: Strategic Blueprint for 2030](https://apartresearch.com/sprints/projects/economics-of-ai-data-center-energy-infrastructure-strategic-blueprint-for-2030-scim): 3rd place by peer review, Economics of Transformative AI. AI data centers are projected to triple U.S. electricity demand by 2030, outpacing the energy sector’s ability to respond. This research… - [Model Models: Simulating a Trusted Monitor](https://apartresearch.com/sprints/projects/model-models-simulating-a-trusted-monitor-r682): 3rd place by peer review, AI Control Hackathon 2025. We offer initial investigations into whether the untrusted model can 'simulate' the trusted monitor: is U able to successfully guess what… - [Adding Document Summaries to Control Arena](https://apartresearch.com/sprints/projects/adding-document-summaries-to-control-arena-fggr): 4th place by peer review, AI Control Hackathon 2025. One of the current focuses of AI Control research is finding new settings within which to examine the control problem. In this hackathon,… - [Can models use their Chain-of-Thought to attack overseers?](https://apartresearch.com/sprints/projects/can-models-use-their-chainofthought-to-attack-overseers-prcv): 1st place by peer review, AI Control Hackathon 2025. This project explores vulnerabilities in AI evaluation mechanisms, specifically focusing on how AI agents might influence their overseeing… - [Schelling coordination via agentic loops](https://apartresearch.com/sprints/projects/schelling-coordination-via-agentic-loops-azvd): 2nd place by peer review, AI Control Hackathon 2025. Capability to collude undermines untrusted monitoring protocols. We propose a novel way to evaluate acausal coordination capabilities of… - [Detecting Malicious AI Agents Through Simulated Interactions](https://apartresearch.com/sprints/projects/detecting-malicious-ai-agents-through-simulated-interactions): Social Sciences Track Prize, Women in AI Safety Hackathon. This research investigates malicious AI Assistants’ manipulative traits and whether the behaviours of malicious AI Assistants can be… - [Red-teaming with Mech-Interpretability](https://apartresearch.com/sprints/projects/red-teaming-with-mech-interpretability): Mechanistic Interpretability Prize, Women in AI Safety Hackathon. Red teaming large language models (LLMs) is crucial for identifying vulnerabilities before deployment, yet systematically creating… - [Morph: AI Safety Education Adaptable to (Almost) Anyone](https://apartresearch.com/sprints/projects/morph-ai-safety-education-adaptable-to-%28almost%29-anyone): Education Track Prize, Women in AI Safety Hackathon. One-liner: Morph is the ultimate operation stack for AI safety education—combining dynamic localization, policy simulations, and ecosystem… - [AntiMidas: Building Commercially-Viable Agents for Alignment Dataset Generation](https://apartresearch.com/sprints/projects/antimidas-building-commercially-viable-agents-for-alignment-dataset-generation): 1st place by peer review, AI Safety Entrepreneurship Hackathon. AI alignment lacks high-quality, real-world preference data needed to align agentic superintel- ligent systems. Our technical innovation… - [AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control](https://apartresearch.com/sprints/projects/autosteer-weight-preserving-reinforcement-learning-for-interpretable-model-control): 1st place by peer review, Reprogramming AI Models Hackathon. Traditional fine-tuning methods for language models, while effective, often disrupt internal model features that could provide valuable… - [Promoting School-Level Accountability for the Responsible Deployment of AI and Related Systems in K-12 Education: Mitigating Bias and Increasing Transparency](https://apartresearch.com/sprints/projects/promoting-school-level-accountability-for-the-responsible-deployment-of-ai-and-related-systems-in-k-12-education-mitigating-bias-and-increasing-transparency): 1st place by peer review, Howard University AI Safety Summit & Policy Hackathon. This policy memorandum draws attention to the potential for bias and opaqueness in intelligent systems utilized in K–12 education, which… - [Robust Machine Unlearning for Dangerous Capabilities](https://apartresearch.com/sprints/projects/robust-machine-unlearning-for-dangerous-capabilities): 1st place by peer review, AI Policy Hackathon at Johns Hopkins University. We test different unlearning methods to make models more robust against exploitation by malicious actors for the creation of bioweapons. - [Diamonds are Not All You Need](https://apartresearch.com/sprints/projects/diamonds-are-not-all-you-need): 1st place by peer review, Agent Security Hackathon. This project tests an AI agent in a straightforward alignment problem. The agent is given creative freedom within a Minecraft world and is… - [DarkForest - Defending the Authentic and Humane Web](https://apartresearch.com/sprints/projects/darkforest-defending-the-authentic-and-humane-web): 1st place by peer review, Hackathon for Technical AI Safety Startups. DarkForest is a pioneering Human Content Verification System (HCVS) designed to safeguard the authenticity of online spaces in the face of… - [AI Alignment Knowledge Graph](https://apartresearch.com/sprints/projects/ai-alignment-knowledge-graph): 1st place by peer review, Research Augmentation Hackathon. We present a web based interactive knowledge graph with concise topical summaries in the field of AI alignement - [Sandbag Detection through Model Degradation](https://apartresearch.com/sprints/projects/sandbag-detection-through-model-degradation): 1st place by peer review, Deception Detection Hackathon: Preventing AI deception. We propose a novel technique to detect sandbagging in LLMs by adding varying amount of noise to model weights and monitoring performance.