# Apart Research > Apart Research is a nonprofit AI safety organization. We research and evaluate harmful manipulation and other risks in frontier AI systems, and we run Sprints (weekend research hackathons) and Fellowships (mentored research towards publication) that bring new researchers into AI safety. Website: https://apartresearch.com. Contact: contact@apartresearch.com (general and media), sprints@apartresearch.com (Sprint prizes). Last updated: October 22, 2026. Key facts: - Nonprofit, remote-first and globally distributed. CEO: Jaime Raldúa Veuthey. - Research focus: harmful manipulation by advanced AI, one of the systemic risks under the EU AI Act. Our work supports the EU AI Office, including a harmful-manipulation evaluation tender, the Code of Practice, the AI Act Advisory Forum and the Expert Forum on Frontier AI. - Track record: 60+ publications in AI safety (NeurIPS, ICLR, ICML, AAAI, ACL, Science), 100+ researchers in the Apart Fellowship, 8,700+ Sprint participants, 1,600+ Sprint projects, 50+ Sprints since 2022, participants from 120+ nationalities. - We work with frontier AI labs (targeted model-behavior evaluations, independent testing of safeguards), policymakers and public institutions (empirical studies and technical evidence for policy and enforcement), and AI safety researchers (methods, benchmarks, collaborations). How our programs connect: - Apart Fellowship (anyone can start): Sprint → top 5–20% of teams invited to Apart Studio (about 6–8 weeks) → about 40% of Studio teams selected for the Apart Fellowship (6–16 weeks) → published research. No application form. - Partnered Fellowships (apply directly, no Sprint needed): application → selected by the partner's mentors → about 16 weeks on the partner's research agenda → published research. - Both are remote and part-time (minimum 10 hours per week), with compute, travel and publication support. No stipend by default. This file contains the text of Apart's main pages. For a shorter link index, see https://apartresearch.com/llms.txt. For every submitted Sprint project, see https://apartresearch.com/sprints/projects/llms.txt. ## About Apart Research Source: https://apartresearch.com/about Apart Research is a nonprofit AI safety organization. We develop research and evaluations focused on harmful manipulation by advanced AI. Through Sprints and Fellowships, we also help researchers, engineers and domain experts turn promising ideas into rigorous research. Apart is remote-first and globally distributed. ### How we work We investigate priority risks and build the field's capacity to address them. - Research Division: research scientists and evaluation engineers who define risks, develop methods, evaluate advanced AI systems and translate findings into useful evidence. They investigate harmful manipulation, build benchmarks and evaluation methods, evaluate advanced AI models, and publish findings for research and oversight. - Field-building programs: Sprints and Fellowships help researchers, engineers and domain experts investigate concrete safety problems and develop promising ideas into substantive research. ### Impact at a glance - 60+ publications in AI safety - 100+ researchers in the Apart Fellowship - 1,600+ Sprint projects submitted - 8,700+ Sprint participants - 50+ Sprints run - 120+ nationalities in our programs ### Team - Jaime Raldúa Veuthey, CEO. Previously robotics research and software engineering; founded a consultancy helping effective altruism organizations with tech and data. - Jerome Wynne, Research Scientist. Previously developed evaluations and research tools at the UK AI Security Institute and Prolific AI Research, including evaluations for impersonation, sandbox escape and agent misuse; earlier built national security and defence applications at Faculty. - Lorenzo Pacchiardi, Research Scientist. Worked on AI evaluation at the University of Cambridge for three years while advising the European Commission on implementing and enforcing the EU AI Act, most recently as a member of the AI Act Scientific Panel. - Adrian Arnaiz-Rodriguez, Research Scientist. - Akash Kundu, Evaluations Engineer. First-author papers at ICLR (DarkBench, Oral) and NeurIPS; contributed to Singapore IMDA's AI safety red-teaming evaluation report; Cooperative AI Research Fellowship alumnus. - Jaco Du Toit, Evaluations Engineer. ML engineer who has built agentic evaluations for the UK AI Security Institute and collaborated with its Science of Evaluations team. - Kamil Alaa, Sprints Coordinator. R&D consulting background; runs Sprint operations. - Eitán Sprejer, Fellowship Lead. Founded AI Safety Argentina; facilitator and teaching assistant for the Center for AI Safety and ML4Good; leads research teams at Apart. - Moïri Gamboni, AI Infrastructure Engineer. Board: Esben Kran, Buck Shlegeris, Mathias Kirk Bonde, Finn Metz. Advisors: Jason Hoelscher-Obermaier (Research Advisor), Anna Wisakanto (Strategy Advisor). ### Media boilerplate Source: https://apartresearch.com/media-kit Apart Research is a nonprofit working on independent AI safety research and evaluations. Its work examines model behavior, capabilities and risks through empirical evaluations and technical investigation. Apart tests general-purpose AI models for harmful manipulation, one of the systemic risks under the EU AI Act. Alongside its research, Apart runs Sprints and Fellowships that help researchers develop early ideas, collaborate and work toward publications. Media inquiries: contact@apartresearch.com. ## Who we work with Source: https://apartresearch.com/contact - Frontier AI labs and AI providers: targeted model-behavior evaluations, independent testing of safeguards, research on safety-relevant behavior including harmful manipulation, and evidence for internal safety decisions. - Policymakers and public institutions: empirical studies tied to policy questions, reproducible methods, and technical evidence designed to support policy, oversight and enforcement. - Research collaborators: joint studies, benchmarks, evaluation development, replication and other collaborations that strengthen the field's evidence base. - Researchers, engineers and domain experts: Sprints, Fellowships, collaborations and roles in Apart's Research Division. - Funders and program sponsors: support for independent research, model evaluations, Sprints, Fellowships and the wider AI safety field. To get in touch, use https://apartresearch.com/contact and choose a topic: evaluation or testing for an AI provider; policy, regulation or public-sector research; research collaboration; Research Sprints or Fellowships; careers; sponsoring a Sprint or Fellowship; or general inquiry. Do not submit confidential model information, credentials, personal data or restricted materials through the form; Apart can set up an appropriate channel for sensitive discussions. ## Research focus Source: https://apartresearch.com/research Apart Research develops the evidence and evaluation methods needed to identify harmful manipulation in frontier AI systems. This work supports the EU AI Office as it prepares to enforce the EU AI Act. Our involvement spans research published in Science, the Code of Practice, the AI Act Advisory Forum, the Expert Forum on Frontier AI and a harmful-manipulation evaluation tender. Research areas: manipulation, deception, model evaluation, AI governance, dangerous capabilities, and AI agents. Output types: papers, technical reports, policy briefs and workshop papers. ## Programs overview Source: https://apartresearch.com/programs Sprints are weekend research hackathons on one AI safety topic, remote-first and open to anyone in the world. Fellowships are months of mentored research towards a paper: the Apart Fellowship by invitation after a Sprint, and Partnered Fellowships by direct application. Both paths lead to published research at venues like NeurIPS, ICLR and ACL. ## Sprints Source: https://apartresearch.com/sprints Research hackathons, about once a month, online and at local sites. Teams take one open AI safety question and, over a weekend, build something others can check: an experiment, an evaluation, a dataset or a tool. No AI safety background is needed. How a Sprint works: 1. Pick a question from the Sprint's brief, or a well-scoped idea of your own. 2. Form a team: bring one, or find collaborators in Discord (https://discord.gg/dYUWDm7Ben) and at local sites. 3. Build and test an experiment, evaluation, dataset, prototype or analysis, with mentors on hand. 4. Submit the question, method, findings and limitations, written so others can review and build on it. Each Sprint has live mentor Q&A in Discord and starter resources and research ideas for each track. Most Sprints run online, often with in-person local sites; a few are in person only. Recent Sprints offer $2,000 in cash prizes across tracks. Sprint write-ups are reviewed for info hazards during judging, under Apart's Responsible Disclosure Policy. Who takes part: ML and AI safety researchers (hypotheses, evaluation designs, experiments), engineers (infrastructure, model integrations, datasets, tools), domain experts (security, economics, finance, law, biology, psychology, governance) and researchers from other fields. Win a Sprint, get invited: the top 5–20% of teams are invited to Apart Studio to turn their project into a research proposal. About 40% of Studio teams go on to the Apart Fellowship. There is no application form: your project is how you are selected. Sprint FAQ: - Can I join without a team? Yes. Teams form at the start of the Sprint, in Discord and at local sites. - What should a team submit? Something others can review: an experiment, evaluation, dataset, prototype, tool or analysis. - Are Sprints online or in person? Most run online, often with in-person local sites; a few are in person only. Each Sprint's page lists its format and local sites. - Can I host a local site? Yes. Use "Host a local site" on the Sprint's page and Apart will list it. - How are prizes paid? By bank transfer; team prizes are split equally between members. See the prize terms below. - Can Apart support a hackathon we run? Yes. Propose a partner hackathon at https://apartresearch.com/sprints/collaborations/submit. ### Sprint prize terms Source: https://apartresearch.com/sprints/prize-terms 1. Prize payments are made through Apart's fiscal sponsor, the registered 501(c)(3) nonprofit Ashgro Inc. 2. Winners must be able to accept a bank transfer from a US charity. If they cannot (mostly in countries under bank-transfer embargoes), they may nominate the prize to an international registered charity of their choice, including Apart. 3. If a winner does not respond after two follow-ups and at most one month after the first email, the prize is donated back to Apart. An email explaining special circumstances counts as a response. 4. Citizens of countries without a US tax treaty have 30% US tax withheld. Non-US citizens competing in the US agree that Apart will file a form with the IRS for them. 5. Team prizes are split equally among team members. This cannot be changed by team agreement. 6. Teams must submit the prize form within one week. Payment emails come from Mercury, not Ashgro Inc. 7. International recipients (especially Indian bank accounts) must provide SWIFT details compatible with international transfers. Questions about prize payments: sprints@apartresearch.com. ## Fellowships Source: https://apartresearch.com/fellowships Two part-time, remote paths for developing promising AI safety work into rigorous research. Both provide mentorship, structure, compute funding and support toward publication, with a minimum commitment of 10 hours per week. | | Apart Fellowship | Partnered Fellowships | |---|---|---| | How to join | By invitation after a Sprint | Apply directly | | Research idea | Your own, from a Sprint | Set by a partner organization | | Guide | An Apart research project manager | A mentor from the partner organization | | Length | 6–16 weeks (4–8 months from Sprint to paper) | About 16 weeks | ### Apart Fellowship Source: https://apartresearch.com/fellowships/apart-fellowship The Apart Fellowship takes the strongest Sprint projects to published research. 1. Sprint: a weekend research hackathon. Anyone can join. The most promising projects, usually the top 5–20%, are invited to Apart Studio. 2. Apart Studio: by invitation, about 6–8 weeks. Write up the Sprint project, then shape it into a research proposal with guidance and compute from the Apart team. 3. Apart Fellowship: about 40% of Studio teams are selected for 6–16 weeks of research with a research project manager, compute and publication guidance. 4. Publication: teams aim for top venues and their workshops (NeurIPS, ICLR, ACL). Apart covers travel to present the work. Track record: 100+ fellows and 30+ papers at top venues and their workshops, including oral presentations. Fellows have joined organizations such as Anthropic and METR, or founded startups such as Geodesic Research and Asymmetric Security. ### Partnered Fellowships Source: https://apartresearch.com/fellowships/partnered-fellowships Apply directly to a research project set by a partner organization. Partners set the agenda and mentor the teams; Apart runs the program, with project management, compute and publication support. No Sprint needed. 1. Application: read an open fellowship's projects and mentors, then apply on its page with a written application and, for some, a work test or interview. 2. Selection: mentors select participants for each project, with support from the Apart team. 3. Fellowship: a scoped project with direct mentorship, compute, project management and regular feedback, for about 16 weeks, fully remote. 4. Publication: concrete outputs aimed at top venues; Apart supports publication and covers travel to present in person. Track record: the first Partnered Fellowship, with Martian, secured over $70,000 in research support for fellows, and two teams were accepted to NeurIPS 2025 workshops. Organizations that want to run or sponsor a Partnered Fellowship on their research agenda can apply as a partner: https://apartresearch.notion.site/240fcfd1de9d80839b5beb781fe4482d. Researchers who want to mentor Fellowship teams can apply as a mentor: https://apartresearch.notion.site/2b8fcfd1de9d80979154c372abd51522. ### Fellowship FAQ - What is the difference between the two Fellowships? In the Apart Fellowship you bring your own research direction from a Sprint, by invitation. In Partnered Fellowships a partner organization sets the agenda and mentors, Apart runs the program, and you apply directly. - What is Apart Studio? The by-invitation step between a Sprint and the Apart Fellowship. You write up your Sprint project with guidance from experienced researchers, then shape it into a research proposal over about 6–8 weeks. The strongest proposals are selected for the Fellowship. - What is the time commitment? Part-time and remote, a minimum of 10 hours per week, more around submission deadlines. Designed to fit around full-time work. - Who is it for? Technical people anywhere in the world: domain experts from adjacent fields (biology, physics, cybersecurity, economics), professionals exploring AI safety part-time, and students. Current participants have a median of 4.3 years of professional experience; 74% are working professionals and 26% students. No formal credentials are required. - What support do participants receive? A collaborative research environment, help with writing and research engineering, research resources, API credits and cloud compute (e.g. GPUs), and help applying for grants. The Apart Fellowship adds a research project manager, a longer project and closer work with research advisors. - Is it paid? No stipend by default. Both programs cover API credits, compute and travel to present, and Apart helps participants apply for external funding. Some Partnered Fellowships pay stipends or prizes; their pages say so. - Who owns the research? Participants are lead authors on resulting publications. Authorship follows substantial contribution, as in standard academic practice. Participants can list Apart Research as their affiliation. - What research areas? Technical AI safety across many domains: model evaluations, interpretability, multi-agent systems, manipulation and deception, CBRN risks, formal methods, cybersecurity and more. - Will I collaborate with external researchers? Yes, where relevant to the project. ## Responsible disclosure Source: https://apartresearch.com/info/responsible-disclosure-policy Apart is committed to open science, within limits set to avoid spreading information that could enable dangerous use of AI or substantially weaken defenses against AI risks (info hazards). The policy covers the Apart team, Fellowship participants and Sprint participants, in papers, preprints, blog and forum posts, talks and podcasts. Examples of info hazards include a jailbreak more effective than known ones against current state-of-the-art models, techniques that let developers fake good performance on critical safety evaluations, and novel methods for using AI for mass manipulation. Risk levels and actions: - Highest: do not disclose that the project exists. - High: disclose the project's existence but not its findings or code. - Intermediate: publish main findings but redact critical implementation details and code. - Low: publish everything. Sprint write-ups are reviewed by the Sprint jury during judging; Fellowship projects are also reviewed by mentors before any publication. Teams are told in writing if a project is assessed above low risk. The policy can be updated by a majority of Apart's leadership team. ## Work with Apart - [Contact](https://apartresearch.com/contact): Evaluations or testing for an AI provider, policy and public-sector research, research collaborations, Sprints and Fellowships, careers, sponsoring a Sprint or Fellowship, and general inquiries. Please do not send confidential model information, credentials or restricted material through the form; we set up a suitable channel for sensitive discussions. - Funding and donations: Apart is funded by grants and donations. To donate, fund research or sponsor a Sprint or Fellowship, use the contact page (choose "Sponsoring a Sprint or Fellowship" or "General inquiry") or email contact@apartresearch.com. - [Careers](https://apartresearch.com/careers): Open roles and open applications. Currently hiring: [Chief Operating Officer](https://apartresearch.com/careers/chief-operating-officer) (full time, remote (4+ hours overlap with CET working hours preferred)); [Director of Research Programs](https://apartresearch.com/careers/director-of-research-programs) (full-time, remote (4+ hours overlap with CET working hours preferred)); [Special Projects Manager](https://apartresearch.com/careers/special-projects-manager) (full-time or Part-time, remote (4+ hours overlap with CET working hours preferred)); [Fellowships Lead](https://apartresearch.com/careers/fellowships-lead) (full-time, remote (significant working-hour overlap with UTC-6/7 and UTC+1/2)); [AI Cyber Threat Researcher (OSINT)](https://apartresearch.com/careers/ai-cyber-threat-researcher-osint) (full-time or Part-time, remote (4+ hours overlap with CET working hours preferred)); [Research Scientist, Manipulation Evals](https://apartresearch.com/careers/research-scientist-manipulation-evals) (full time, remote, Europe or San Francisco preferred). - Mentors and partners: Mentor Fellowship teams, or run or sponsor a Partnered Fellowship on your research agenda. Apart handles recruiting, project management and compute. [Apply as a mentor](https://apartresearch.notion.site/2b8fcfd1de9d80979154c372abd51522) or [apply as a partner](https://apartresearch.notion.site/240fcfd1de9d80839b5beb781fe4482d). - [Propose a partner hackathon](https://apartresearch.com/sprints/collaborations/submit): Apart can host your AI safety hackathon's page, sign-ups and project submissions. ## Publications - [Research](https://apartresearch.com/research): Apart's publications and benchmarks, with filters by research area (manipulation, deception, model evaluation, AI governance, dangerous capabilities, AI agents). - [All publications on Google Scholar](https://scholar.google.com/citations?hl=en&user=f2-xhQkAAAAJ): Apart Research's full publication list. - [The science and practice of proportionality in AI risk evaluations](https://doi.org/10.1126/science.aea3835): Science 2026, Policy Forum. AI evaluations should provide meaningful risk information without imposing excessive burden. - [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://proceedings.neurips.cc/paper_files/paper/2025/hash/aa95d031f5e151bf8fb402f44bd97054-Abstract-Conference.html): NeurIPS 2025. Detects hidden capabilities by testing how models respond to noise in their weights. - [DarkBench: Benchmarking Dark Patterns in Large Language Models](https://arxiv.org/abs/2503.10728): ICLR 2025, Oral. A benchmark of 660 prompts for manipulation, sycophancy and other dark patterns in leading language models. Benchmark site: https://darkbench.ai/ - [Multi-Agent Risks from Advanced AI](https://arxiv.org/abs/2502.14143): Cooperative AI Foundation 2025, technical report. Maps the failure modes that emerge when many AI agents interact. - [Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systems](https://openreview.net/forum?id=ljBR8i3jmX): AAAI 2025. - [Rethinking CyberSecEval: An LLM-Aided Approach to Evaluation Critique](https://arxiv.org/abs/2411.08813): NeurIPS 2024 workshop. - [CryptoFormalEval: Integrating LLMs and Formal Verification for Automated Cryptographic Protocol Vulnerability Detection](https://openreview.net/forum?id=hqvJT2pOxu): NeurIPS 2024 workshop. - [Sandbag Detection through Model Impairment](https://openreview.net/forum?id=zpcx5a8w2a): NeurIPS 2024 workshop. - [Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities](https://arxiv.org/abs/2410.09114): arXiv preprint 2024. - [Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts](https://arxiv.org/abs/2410.09247): ICML 2024 workshop. - [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://arxiv.org/abs/2401.05566): arXiv preprint 2024. An Anthropic-led paper; Apart contributed as a collaborator. - [Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark](https://aclanthology.org/2023.findings-acl.733/): Findings of ACL 2023. - [Responsible Disclosure Policy](https://apartresearch.com/info/responsible-disclosure-policy): How Apart reviews and limits publication of dual-use research, including Sprint projects. ## All Sprints Source: https://apartresearch.com/sprints/all Newest first, with the number of published projects where available. - [Threat Intelligence & Threat Modelling Hackathon](https://apartresearch.com/sprints/threat-intelligence-threat-modelling-2026): Upcoming, Dec 4–6, 2026. Can LLMs scale up threat intelligence and threat modelling for AI data centres facing nation-state attackers? With Security Level 5 (SL5). - [Agent Attribution Hackathon](https://apartresearch.com/sprints/agent-attribution-2026): Upcoming, Nov 27–29, 2026. Telling rogue AI agents apart from human-steered ones. One weekend to build attribution classifiers, simulate agents that evade them, and find rogue agents in the wild. - [AI x Epistemics Research Sprint](https://apartresearch.com/sprints/ai-x-epistemics-research-sprint-2026-11-13-to-2026-11-15): Upcoming, Nov 13–15, 2026. AI for epistemics: whether models know how solid their claims are, trust and provenance infrastructure, and products that improve decisions. Online. - [Swarm Alignment Hackathon](https://apartresearch.com/sprints/swarm-alignment-2026): Upcoming, Nov 6–8, 2026. Alignment can't stop at single models when AI agents start to negotiate, trade and share tools. Spend a weekend uncovering and preventing failures that emerge between multiple agents. - [AI Collusion Research Sprint](https://apartresearch.com/sprints/ai-collusion-research-sprint-2026-10-23-to-2026-10-25): Upcoming, Oct 23–25, 2026. Collusion between AI agents in markets and workflows: detecting, auditing and breaking it. Co-organized with Poseidon Research and AE Studio, supported by Modal and the AI Alignment Foundation; online with hubs at Collider (New York City) and AI Safety Hong Kong. - [AI Incident Response Sprint](https://apartresearch.com/sprints/ai-incident-response-sprint-2026-09-11-to-2026-09-13): Past, Sep 11–13, 2026. Incident response for autonomous AI agents, using the public record of the July 2026 OpenAI–Hugging Face agent intrusion. 231 projects. - [Digital Minds Research Sprint](https://apartresearch.com/sprints/digital-minds-research-sprint-2026-08-14-to-2026-08-16): Past, Aug 14–16, 2026. Preferences, welfare signals, introspection and identity in frontier AI models. 233 projects. - [Secret Loyalties Hackathon](https://apartresearch.com/sprints/secret-loyalties-hackathon-2026-07-24-to-2026-07-26): Past, Jul 24–26, 2026. Secretly loyal AI: models that covertly advance a hidden principal's interests while appearing normal. 177 projects. - [Global South AI Safety Hackathon](https://apartresearch.com/sprints/global-south-ais-hackathon-2026-06-19-to-2026-06-21): Past, Jun 19–21, 2026. First AI safety hackathon designed for and with the Global South, across 7 regional hubs in Latin America, Africa and Asia. 216 projects. - [The Secure Program Synthesis Hackathon](https://apartresearch.com/sprints/secure-program-synthesis-hackathon-2026-05-22-to-2026-05-24): Past, May 22–24, 2026. AI, formal methods and security: tools that catch what AI code generation gets wrong. Co-organized with Atlas Computing. 61 projects. - [AIxBio Hackathon](https://apartresearch.com/sprints/aixbio-hackathon-2026-04-24-to-2026-04-26): Past, Apr 24–26, 2026. Biosecurity tools at the intersection of AI and biology, such as DNA synthesis screening and pandemic early warning. - [AI Control Hackathon 2026](https://apartresearch.com/sprints/ai-control-hackathon-2026-03-20-to-2026-03-22): Past, Mar 20–22, 2026. AI control: techniques that keep untrusted AI systems safe to deploy, including monitoring and sabotage detection. 125 projects. - [The Technical AI Governance Challenge](https://apartresearch.com/sprints/the-technical-ai-governance-challenge-2026-01-30-to-2026-02-01): Past, Jan 30–Feb 1, 2026. Verification systems and technical infrastructure for international AI governance. 47 projects. - [AI Manipulation Hackathon](https://apartresearch.com/sprints/ai-manipulation-hackathon-2026-01-09-to-2026-01-11): Past, Jan 9–11, 2026. Measurement and defense tools against AI manipulation. 70 projects. - [Defensive Acceleration Hackathon](https://apartresearch.com/sprints/def-acc-hackathon-2025-11-21-to-2025-11-23): Past, Nov 21–23, 2025. Societal defenses against AI-enabled threats. 83 projects. - [The AI Forecasting Hackathon](https://apartresearch.com/sprints/the-ai-forecasting-hackathon-2025-10-31-to-2025-11-02): Past, Oct 31–Nov 2, 2025. Forecasting AI progress and its impacts. 34 projects. - [ARENA 6.0 Mechanistic Interpretability Hackathon](https://apartresearch.com/sprints/arena-6-mechanistic-interpretability-hackathon-2025-09-13-to-2025-09-14): Past, Sep 13–14, 2025. Mechanistic interpretability projects with the ARENA 6.0 cohort. - [CBRN AI Risks Research Sprint](https://apartresearch.com/sprints/cbrn-ai-risks-sprint-2025-09-12-to-2025-09-14): Past, Sep 12–14, 2025. Chemical, biological, radiological and nuclear risks from AI. - [AI Safety x Physics Grand Challenge](https://apartresearch.com/sprints/ai-safety-x-physics-grand-challenge-2025-07-25-to-2025-07-27): Past, Jul 25–27, 2025. Applying physics methods to AI safety problems. 21 projects. - [Red Teaming A Narrow Path: ControlAI Policy Sprint](https://apartresearch.com/sprints/red-teaming-a-narrow-path-control-ai-policy-sprint-2025-06-13-to-2025-06-13): Past, Jun 13, 2025. Stress-testing ControlAI's "A Narrow Path" policy proposals. 20 projects. - [Apart x Martian Mechanistic Router Interpretability Hackathon](https://apartresearch.com/sprints/apart-x-martian-mechanistic-router-interpretability-hackathon-2025-05-30-to-2025-06-01): Past, May 30–Jun 1, 2025. Interpretable judge models and routing for expert orchestration, with Martian. 19 projects. - [Economics of Transformative AI](https://apartresearch.com/sprints/economics-of-transformative-ai-research-sprint-2025-04-25-to-2025-04-27): Past, Apr 25–27, 2025. Economic effects of transformative AI. 12 projects. - [Dark Patterns in AGI Hackathon at ZAIA](https://apartresearch.com/sprints/dark-patterns-in-agi-hackathon-at-zaia-2025-04-04-to-2025-04-05): Past, Apr 4–6, 2025. Dark patterns and manipulation in AI systems, with the ZAIA community. 3 projects. - [AI Control Hackathon 2025](https://apartresearch.com/sprints/ai-control-hackathon-2025-03-29-to-2025-03-30): Past, Mar 29–30, 2025. AI control protocols and settings. 19 projects. - [Women in AI Safety Hackathon](https://apartresearch.com/sprints/women-in-ai-safety-hackathon-2025-03-07-to-2025-03-14): Past, Mar 7–10, 2025. AI safety hackathon highlighting women in the field; Apart's biggest event at the time. 33 projects. - [AI Safety Entrepreneurship Hackathon](https://apartresearch.com/sprints/ai-safety-entrepreneurship-hackathon-2025-01-17-to-2025-01-20): Past, Jan 17–20, 2025. Building AI safety startups. 19 projects. - [Reprogramming AI Models Hackathon](https://apartresearch.com/sprints/reprogramming-ai-models-hackathon-2024-11-22-to-2024-11-25): Past, Nov 22–25, 2024. Interpretability and feature steering, with Goodfire's API. 28 projects. - [AI Policy Hackathon at Johns Hopkins University](https://apartresearch.com/sprints/ai-policy-hackathon-at-johns-hopkins-university-2024-10-26-to-2024-10-28): Past, Oct 26–28, 2024. AI policy proposals, hosted at Johns Hopkins University. 29 projects. - [Agent Security Hackathon](https://apartresearch.com/sprints/agent-security-hackathon-2024-10-04-to-2024-10-07): Past, Oct 4–7, 2024. Security of LLM agents. 12 projects. - [ARENA 4.0 Interpretability Hackathon](https://apartresearch.com/sprints/arena-4-interp-hackathon-2024-09-15-to-2024-09-15): Past, Sep 15, 2024. Interpretability projects with the ARENA 4.0 cohort. 4 projects. - [The Concordia Contest](https://apartresearch.com/sprints/the-concordia-contest-2024-09-06-to-2024-09-09): Past, Sep 6–9, 2024. Cooperative intelligence of language model agents. 5 projects. - [Hackathon for Technical AI Safety Startups](https://apartresearch.com/sprints/hackathon-for-technical-ai-safety-startups-2024-09-22-to-2024-09-25): Past, Aug 30–Sep 2, 2024. Technical AI safety startup ideas. 12 projects. - [AI capabilities and risks demo-jam](https://apartresearch.com/sprints/ai-capabilities-and-risks-demo-jam-2024-08-23-to-2024-08-26): Past, Aug 23–26, 2024. Interactive demonstrations of AI capabilities and risks. 17 projects. - [Research Augmentation Hackathon](https://apartresearch.com/sprints/research-augmentation-hackathon-2024-07-26-to-2024-07-29): Past, Jul 26–29, 2024. Tools that speed up AI alignment research. 9 projects. - [Deception Detection Hackathon: Preventing AI deception](https://apartresearch.com/sprints/deception-detection-hackathon-preventing-ai-deception-2024-06-28-to-2024-07-01): Past, Jun 28–Jul 1, 2024. Detecting deception in AI systems, with Apollo Research. 21 projects. - [Computational Mechanics Hackathon](https://apartresearch.com/sprints/computational-mechanics-hackathon-2024-06-01-to-2024-06-03): Past, Jun 1–3, 2024. Interpretability through computational mechanics. 8 projects. - [AI Security Evaluation Hackathon: Measuring AI Capability](https://apartresearch.com/sprints/ai-security-evaluation-hackathon-2024-05-24-to-2024-05-27): Past, May 24–27, 2024. Benchmarks that measure dangerous AI capabilities. 12 projects. - [AI and Democracy Hackathon: Demonstrating the Risks](https://apartresearch.com/sprints/ai-and-democracy-hackathon-2024-05-03-to-2024-05-06): Past, May 3–6, 2024. Demonstrating and mitigating AI risks to democracy. 25 projects. - [InterpVis](https://apartresearch.com/sprints/interpvis-2024-03-12-to-2024-03-14): Past, Apr 13–14, 2024. Visualizations for mechanistic interpretability, at the LISA offices in London. - [LLM Psychology Hackathon](https://apartresearch.com/sprints/llm-psychology-hackathon-2024-04-05-to-2024-04-07): Past, Apr 5–7, 2024. Studying LLM behavior with methods from psychology. - [Code Red Hackathon: Can LLMs Replicate and do R&D by Themselves?](https://apartresearch.com/sprints/code-red-hackathon-2024-03-22-to-2024-03-25): Past, Mar 22–25, 2024. Evaluation tasks for autonomous replication and AI R&D, with METR. - [Multi-Agent Security Research Sprint](https://apartresearch.com/sprints/multi-agent-security-research-sprint-2024-02-09-to-2024-02-11): Past, Feb 9–11, 2024. Securing multi-agent systems in deployment. - [The AI Governance Research Sprint](https://apartresearch.com/sprints/the-ai-governance-research-sprint-2024-01-05-to-2024-05-07): Past, Jan 5–7, 2024. AI governance research. - [AI Model Evaluations Hackathon](https://apartresearch.com/sprints/ai-model-evaluations-hackathon-2023-11-24-to-2023-11-26): Past, Nov 24–26, 2023. Evaluations of AI model behavior. - [Multi-Agent Safety Hackathon](https://apartresearch.com/sprints/multi-agent-safety-hackathon-2024-09-29-to-2023-11-01): Past, Sep 29–Oct 1, 2023. Multi-agent safety, with the Cooperative AI Foundation. - [The Agency Foundations Challenge](https://apartresearch.com/sprints/the-agency-foundations-challenge-2023-09-08-to-2023-09-24): Past, Sep 8–24, 2023. Safeguarding human agency against advanced AI. - [Distillation Write-a-thon 2.0](https://apartresearch.com/sprints/distillation-write-a-thon-2023-08-24-to-2023-08-26): Past, Aug 24–26, 2023. Writing accessible explanations of AI safety research. - [LLM Evals Hackathon](https://apartresearch.com/sprints/llm-evals-hackathon-2023-08-18-to-2023-08-20): Past, Aug 18–20, 2023. Evaluating risks of deployed language models. - [Interpretability Hackathon 3.0](https://apartresearch.com/sprints/interpretability-hackathon-2023-07-14-to-2023-07-16): Past, Jul 14–16, 2023. Interpretability of language models. - [Safety Benchmarks Hackathon](https://apartresearch.com/sprints/safety-benchmarks-hackathon-2023-06-30-to-2023-07-02): Past, Jun 30–Jul 2, 2023. Safety benchmarks for large models. - [Distillation Write-a-thon](https://apartresearch.com/sprints/ai-governance-hackathon-2023-06-16-to-2023-06-18): Past, Jun 16–18, 2023. Writing accessible explanations of AI safety research. - [ML Verifiability Hackathon](https://apartresearch.com/sprints/ml-verifiability-hackathon-2023-05-26-to-2023-05-28): Past, May 26–28, 2023. Formal and informal verification of ML systems. - [The Interpretability Hackathon 2.0](https://apartresearch.com/sprints/the-interpretability-hackathon-2023-04-14-to-2023-04-17): Past, Apr 14–17, 2023. Mechanistic interpretability. - [AI Governance](https://apartresearch.com/sprints/ai-governance-2023-03-24-to-2023-03-27): Past, Mar 24–27, 2023. AI governance and society. - [Scale Oversight for Machine Learning Hackathon](https://apartresearch.com/sprints/scale-oversight-for-machine-learning-hackathon-2023-02-10-to-2023-02-13): Past, Feb 10–13, 2023. Measuring and monitoring the safety of large ML models. - [Mechanistic Interpretability Hackathon](https://apartresearch.com/sprints/mechanistic-interpretability-hackathon-2023-01-20-to-2023-01-23): Past, Jan 20–23, 2023. Mechanistic interpretability. - [EAGx LatAm Epoch AI Hackathon](https://apartresearch.com/sprints/eagx-latam-epoch-ai-hackathon-2023-01-09-to-2023-01-11): Past, Jan 9–11, 2023. Trends in AI development, with Epoch AI, after EAGx LatAm. - [AI Testing](https://apartresearch.com/sprints/ai-testing-2022-12-15-to-2022-12-17): Past, Dec 15–17, 2022. Testing AI systems. - [The Interpretability Hackathon](https://apartresearch.com/sprints/the-interpretability-hackathon-2022-11-11-to-2022-11-13): Past, Nov 11–13, 2022. Interpretability. - [Language Model Hackathon](https://apartresearch.com/sprints/language-model-hackathon-2022-09-29-to-2022-10-01): Past, Sep 29–Oct 1, 2022. Language model behavior (the first Alignment Jam). ## Partnered Fellowships No Partnered Fellowship is taking applications right now. - [The Heron AI Security Fellowship](https://apartresearch.com/fellowships/the-ai-security-fellowship): Finished, January–April 2026. With Heron AI Security; cybersecurity professionals working with AI security field leaders. - [The Secure Program Synthesis Fellowship](https://apartresearch.com/fellowships/the-secure-program-synthesis-fellowship): Finished, June–September 2026. Mentor-led projects on formal methods, AI systems and security. - [The Martian Fellowship on Mechanistic Interpretability](https://apartresearch.com/fellowships/the-martian-fellowship-on-mechanistic-interpretability): Finished, September–December 2025. With Martian; over $70,000 in research support for fellows and two teams accepted to NeurIPS 2025 workshops. ## Partner hackathons - [Berkeley AI Policy Hackathon](https://apartresearch.com/sprints/collaborations/berkeley-ai-policy-hackathon-2025-04-14-to-2025-04-26): Apr 14–26, 2025. Student policy proposals on AI, in California and beyond. - [Georgia Tech AISI Policy Hackathon](https://apartresearch.com/sprints/collaborations/georgia-tech-aisi-policy-hackathon-2025-04-05-to-2025-04-06): Apr 5–6, 2025. AI governance hackathon organized by Georgia Tech. - [Autostructures: interfaces not between humans and AI, but between humans via AI](https://apartresearch.com/sprints/collaborations/blackpoolautostructuring-2024-11-23-to-2024-11-25): Nov 23–25, 2024. Design for infrastructure embedded with AI. - [Howard University AI Safety Summit & Policy Hackathon](https://apartresearch.com/sprints/collaborations/howard-university-ai-safety-summit-policy-hackathon-2024-11-21-to-2024-11-22): Nov 21–22, 2024. AI governance and safety policy, organized by Howard University. ## Local Sprint sites - [AI Alignment Bangalore](https://apartresearch.com/sprints/locations/ai-alignment-bangalore): Bangalore, India - [AI Safety Arc](https://apartresearch.com/sprints/locations/ai-safety-arc): NIT Agartala, India - [AI safety initiative at Georgia Tech](https://apartresearch.com/sprints/locations/ai-safety-initiative-at-georgia-tech): Atlanta, GA, United States - [CEEALAR](https://apartresearch.com/sprints/locations/ceealar): Blackpool, United Kingdom - [Effective Altruism Denmark](https://apartresearch.com/sprints/locations/effective-altruism-denmark): Copenhagen, Denmark - [Effective Altruism Singapore](https://apartresearch.com/sprints/locations/effective-altruism-singapore): Singapore - [Singapore AI Safety Hub](https://apartresearch.com/sprints/locations/singapore-ai-safety-hub): Singapore - [Technology and Policy Society at Johns Hopkins](https://apartresearch.com/sprints/locations/technology-and-policy-society-at-johns-hopkins): Baltimore, Maryland, United States ## Organization - [Home](https://apartresearch.com/): Apart's research on AI manipulation, programs, and how to work with us. - [About](https://apartresearch.com/about): Mission, how we work, impact numbers, team, board and advisors. - [News](https://apartresearch.com/news): Research releases, Sprint round-ups, researcher spotlights and the newsletter archive. - [Media kit](https://apartresearch.com/media-kit): Boilerplate, background and event photos for journalists. Media inquiries: contact@apartresearch.com. - [Discord](https://discord.gg/dYUWDm7Ben): Community for forming Sprint teams, mentor Q&A and discussing research. - Elsewhere: [GitHub](https://github.com/apartresearch), [X](https://x.com/apartresearch), [LinkedIn](https://www.linkedin.com/company/apartresearch/), [YouTube](https://www.youtube.com/channel/UCnfBOJnTkE9sgjMOOsQbi2w). ## Selected news - [Explaining the Apart Research Fellowships](https://apartresearch.com/news/explaining-the-apart-research-fellowships): Dec 3, 2025. How the Apart Fellowship works, and the launch of Partnered Fellowships. - [Problem Areas in Physics and AI Safety](https://apartresearch.com/news/problem-areas-in-physics-and-ai-safety): Jul 25, 2025. Five problem areas where physics methods can help AI safety. - [Beyond Monolithic AI: The Case for an Expert Orchestration Architecture](https://apartresearch.com/news/beyond-monolithic-ai-the-case-for-an-expert-orchestration-architecture): May 14, 2025. Routing queries across many specialized models instead of one monolithic model. - [Engineering a World Designed for Safe Superintelligence](https://apartresearch.com/news/engineering-a-world-designed-for-safe-superintelligence): Apr 9, 2025. Esben Kran's talk at the Paris AI Action Summit. - [Women in AI Safety: Hackathon Round-Up](https://apartresearch.com/news/women-in-ai-safety-hackathon-round-up): Apr 3, 2025. Winners of Apart's largest Sprint at the time. - [Uncovering Model Manipulation with DarkBench](https://apartresearch.com/news/uncovering-model-manipulation-with-darkbench): Feb 18, 2025. The DarkBench benchmark for manipulative dark patterns in popular LLMs. - [Studio Progress Report](https://apartresearch.com/news/studio-progress-report): Feb 13, 2025. Results from the first Apart Studio cohort. - [AI Hackers in the Wild: LLM Agent Honeypot](https://apartresearch.com/news/ai-hackers-in-the-wild-llm-agent-honeypot): Dec 31, 2024. A honeypot for detecting LLM-based attackers in the wild. - [Apart in 2024](https://apartresearch.com/news/apart-in-2024): Jan 1, 2025. Review of Apart's 2024. - [‘3cb’: The Catastrophic Cyber Capabilities Benchmark](https://apartresearch.com/news/3cb-the-catastrophic-cyber-capabilities-benchmark): Oct 30, 2024. A benchmark for LLM agent cyber-offense capabilities. - [Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts](https://apartresearch.com/news/benchmark-inflation-revealing-llm-performance-gaps-using-retro-holdouts): Oct 18, 2024. Public benchmarks can overstate performance because test data leaks into training data. - [The ultimate guide to AI safety research hackathons](https://apartresearch.com/news/the-ultimate-guide-to-ai-safety-research-hackathons): May 17, 2024. How to get the most out of a research Sprint. - [How to organize a research hackathon](https://apartresearch.com/news/how-to-organize-a-research-hackathon): Mar 1, 2024. Guide for local organizers. ## Policies - [Responsible Disclosure Policy](https://apartresearch.com/info/responsible-disclosure-policy): How Apart handles dual-use research and info hazards. - [Code of Conduct](https://apartresearch.com/info/code-of-conduct): Expected conduct at Apart events and in the community. - [Privacy Policy](https://apartresearch.com/info/privacy-policy): How Apart collects, uses and protects personal data from program participants. ## Selected award-winning Sprint projects Every project, grouped by Sprint: https://apartresearch.com/sprints/projects/llms.txt - [Project Anchored](https://apartresearch.com/sprints/projects/project-anchored-q4fn): 2nd Place Winner, Digital Minds Research Sprint. Anchoring vignettes are the standard survey-methodology fix for self-reports that are not comparable across respondents. This project… - [Readable but Not Causal: Limits of Self-Attributed Welfare Representations in Language Models](https://apartresearch.com/sprints/projects/readable-but-not-causal-limits-of-selfattributed-welfare-representations-in-language-models-ehin): 1st Place Winner, Digital Minds Research Sprint. Welfare-like internal representations are increasingly studied as candidate evidence about AI systems. Their entity attribution—whether a… - [One Dial, Not a Tree: Occupational Personas and Emergent Misalignment](https://apartresearch.com/sprints/projects/one-dial-not-a-tree-occupational-personas-and-emergent-misalignment-ukf6): 4th Place Winner, Digital Minds Research Sprint. Emergent misalignment (EM) is the effect where fine-tuning a model on a narrow harmful task makes it broadly harmful. It is already known… - [Sisyphus in the loop: What Makes an LLM Persist?](https://apartresearch.com/sprints/projects/sisyphus-in-the-loop-what-makes-an-llm-persist-yqd9): 5th Place Winner, Digital Minds Research Sprint. As large language models (LLMs) are increasingly deployed as agents that pursue goals over extended horizons, understanding what determines… - [Model, Instance, or Persona? Measuring Affective Signals in Public Text After an AI Is Retired](https://apartresearch.com/sprints/projects/model-instance-or-persona-measuring-aective-signals-in-public-text-after-an-ai-is-retired-hj7u): 3rd Place Winner, Digital Minds Research Sprint. This sprint asks whether the assistant identifies as a model, an instance, or a persona. I ask which of the three its users name. When a… - [Omission Attacks: When Doing Nothing Is the Attack](https://apartresearch.com/sprints/projects/omission-attacks-when-doing-nothing-is-the-attack-0y1v): 1st Place Winner + LinuxArena Track Winner, AI Control Hackathon 2026. AI control protocols monitor agent actions to detect sabotage, but omission attacks exploit what the agent fails to do rather than what it… - [Syscon: Kernel-Level Syscall Monitoring for AI Agent Control in Sandboxed Environments](https://apartresearch.com/sprints/projects/syscon-kernellevel-syscall-monitoring-for-ai-agent-control-in-sandboxed-environments-guun): LinuxArena Track Winner, AI Control Hackathon 2026. Many existing AI control protocols rely on a trusted monitor to evaluate an untrusted agent's actions, but these monitors generally access… - [Detecting LLM Subversion in Vulnerability Patching Settings](https://apartresearch.com/sprints/projects/detecting-llm-subversion-in-vulnerability-patching-settings-gagj): 2nd Place Winner, AI Control Hackathon 2026. LLMs are increasingly used to propose fixes to vulnerabilities in code. If the LLM is misaligned or untrustworthy, it may propose fixes… - [Undoing the Fix: CVE Regression as an AI Control Setting](https://apartresearch.com/sprints/projects/undoing-the-fix-cve-regression-as-an-ai-control-setting-f855): 4th Place Winner, AI Control Hackathon 2026. Production codebases carry accumulated security fixes that can be lost when an AI agent modifies code, whether deliberately or by mistake.… - [Counterfactual Components for AI Control Protocols](https://apartresearch.com/sprints/projects/counterfactual-components-for-ai-control-protocols-ibli): 5th Place Winner, AI Control Hackathon 2026. Current AI control protocols rely heavily on monitors scoring suspiciousness — a single point of failure vulnerable to prompt injection and… - [ActionLens: Pre-Execution Environment Probing for Agent Action Approval](https://apartresearch.com/sprints/projects/actionlens-preexecution-environment-probing-for-agent-action-approval-xy22): 3rd Place Winner, AI Control Hackathon 2026. ActionLens is a pre-execution control protocol for shell and file actions proposed by AI agents. Instead of approving an action from… - [Markov Chain Lock Watermarking: Provably Secure Authentication for LLM Outputs](https://apartresearch.com/sprints/projects/markov-chain-lock-watermarking-provably-secure-authentication-for-llm-outputs-l8oe): 2nd Place Winner, The Technical AI Governance Challenge. We present Markov Chain Lock (MCL) watermarking, a cryptographically secure framework for authenticating LLM outputs. MCL constrains token… - [Prototyping an Embedded Off-Switch for AI Compute](https://apartresearch.com/sprints/projects/prototyping-an-embedded-offswitch-for-ai-compute-65pz): 5th Place Winner, The Technical AI Governance Challenge. This project prototypes an embedded off-switch for AI accelerators. The security block requires periodic cryptographic authorization to… - [Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors](https://apartresearch.com/sprints/projects/fingerprinting-all-ai-cluster-io-without-mutually-trusted-processors-d81l): 4th Place Winner, The Technical AI Governance Challenge. We design and simulate a "border patrol" device for generating cryptographic evidence of data traffic entering and leaving an AI cluster,… - [Political Intelligence for AI Safety: The AI Risk Attitudes Survey (AIRAS)](https://apartresearch.com/sprints/projects/political-intelligence-for-ai-safety-the-ai-risk-attitudes-survey-airas-dlpz): 3rd Place Winner, The Technical AI Governance Challenge. The AI safety and governance community is making progress on defining red lines around existential risk from advanced AI systems, and… - [LidaSim: Testing AI Policies With Persona-Based Simulations](https://apartresearch.com/sprints/projects/lidasim-testing-ai-policies-with-personabased-simulations-o8x6): 1st Place Winner, The Technical AI Governance Challenge. We simulate well-known figures in AI and politics with agents, scraping large amounts of data to get realistic simulations. Then, we test… - [Eliciting Deception on Generative Search Engines](https://apartresearch.com/sprints/projects/eliciting-deception-on-generative-search-engines-6nuu): 2nd Place Winner, AI Manipulation Hackathon. Large language models (LLMs) with web browsing capabilities are vulnerable to adversarial content injection—where malicious actors embed… - [SycophantSee - Activation-based diagnostics for prompt engineering: monitoring sycophancy at prompt and generation time](https://apartresearch.com/sprints/projects/sycophantsee-activationbased-diagnostics-for-prompt-engineering-monitoring-sycophancy-at-prompt-and-generation-time-ys27): 5th Place Winner, AI Manipulation Hackathon. Activation monitoring reveals that prompt framing affects a model's internal state before generation begins. - [Who Does Your AI Serve? Manipulation By and Of AI Assistants](https://apartresearch.com/sprints/projects/who-does-your-ai-serve-manipulation-by-and-of-ai-assistants-77xx): 1st Place Winner, AI Manipulation Hackathon. AI assistants can be both instruments and targets of manipulation. In our project, we investigated both directions across three studies. AI… - [Cross-Linguistic Sycophancy in Frontier LLMs: A Benchmark Study](https://apartresearch.com/sprints/projects/crosslinguistic-sycophancy-in-frontier-llms-a-benchmark-study-w55u): 3rd Place Winner, AI Manipulation Hackathon. We developed a cross-linguistic sycophancy benchmark testing whether frontier AI models exhibit different manipulation behaviours across… - [Agent Attacks via Memory Injection](https://apartresearch.com/sprints/projects/agent-attacks-via-memory-injection-izfz): 4th Place Winner, AI Manipulation Hackathon. LLMs have recently adopted persistent memory to provide models with better user knowledge and personalisation. However, this introduces a… - [Local Learning Coefficients Predict Developmental Milestones During Group Relative Policy Optimization](https://apartresearch.com/sprints/projects/local-learning-coefficients-predict-developmental-milestones-during-group-relative-policy-optimization-2te2): Prize winner, AI Safety x Physics Grand Challenge. In this work, we investigate the emergence of capabilities in reinforcement learning (RL) by framing them as developmental phase… - [AI agentic system epidemiology](https://apartresearch.com/sprints/projects/ai-agentic-system-epidemiology-b83r): Prize winner, AI Safety x Physics Grand Challenge. As AI systems scale into decentralized, multi-agent deployments, emergent vulnerabilities challenge our ability to evaluate and manage… - [Momentum–Point-Perplexity Mechanics in Large Language Models=](https://apartresearch.com/sprints/projects/momentumpointperplexity-mechanics-in-large-language-models-v1nc): Prize winner, AI Safety x Physics Grand Challenge. This work analyzes the hidden states of twenty different open-source transformer language models, ranging from small to medium size and… - [Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process](https://apartresearch.com/sprints/projects/constrained-belief-updates-and-geometric-structures-in-transformer-representations-for-the-rrxor-process-gsok): Prize winner, AI Safety x Physics Grand Challenge. In this work, we seek to further understand the computational structure formed by transformers trained to predict the next token of a data… - [Manipulating Self-Preference for Large Language Models](https://apartresearch.com/sprints/projects/manipulating-selfpreference-for-large-language-models-c4wm): 1st place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. Large language models (LLMs) carry great value as evaluators of synthetic data for research and production settings. However, recent… - [Approximating Human Preferences Using a Multi-Judge Learned System](https://apartresearch.com/sprints/projects/approximating-human-preferences-using-a-multijudge-learned-system-v3im): 2nd place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. In this work, we introduced a learned approach to aggregating multi-judge scores: using a GAM and a simple MLP as an alternative to… - [Guardian-Loop: Mechanistically Interpretable Micro-Judges with Adversarial Self-Improvement](https://apartresearch.com/sprints/projects/guardianloop-mechanistically-interpretable-microjudges-with-adversarial-selfimprovement-1b2p): 4th place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. Guardian-Loop is a mechanistically interpretable judge system designed to enhance the Expert Orchestration Architecture through transparent… - [Judge using SAE Features](https://apartresearch.com/sprints/projects/judge-using-sae-features-y1r1): 3rd place by peer review, Apart x Martian Mechanistic Router Interpretability Hackathon. The key idea of this project was to explore model judgement using Sparse Autoencoder (SAE) features for mathematical reasoning tasks… - [Economic Feasibility of Universal High Income (UHI) in an Age of Advanced Automation](https://apartresearch.com/sprints/projects/economic-feasibility-of-universal-high-income-uhi-in-an-age-of-advanced-automation-jn3g): 4th place by peer review, Economics of Transformative AI. This paper analyzes five interlinked fiscal measures proposed to fund a Universal High Income (UHI) system in response to large-scale… - [Evaluating the risk of job displacement by transformative AI automation in developing countries: A case study on Brazil](https://apartresearch.com/sprints/projects/evaluating-the-risk-of-job-displacement-by-transformative-ai-automation-in-developing-countries-a-case-study-on-brazil-829h): 3rd place by peer review, Economics of Transformative AI. In this paper, we introduce an empirical and reproducible approach to monitoring job displacement by TAI. We first classify occupations… - [The Early Economic Impacts of Transformative AI: A Focus on Temporal Coherence](https://apartresearch.com/sprints/projects/the-early-economic-impacts-of-transformative-ai-a-focus-on-temporal-coherence-ipql): 1st place by peer review, Economics of Transformative AI. We investigate the economic potential of Transformative AI, focusing on "temporal coherence"—the ability to maintain goal-directed behavior… - [Economics of AI Data Center Energy Infrastructure: Strategic Blueprint for 2030](https://apartresearch.com/sprints/projects/economics-of-ai-data-center-energy-infrastructure-strategic-blueprint-for-2030-scim): 3rd place by peer review, Economics of Transformative AI. AI data centers are projected to triple U.S. electricity demand by 2030, outpacing the energy sector’s ability to respond. This research… - [Model Models: Simulating a Trusted Monitor](https://apartresearch.com/sprints/projects/model-models-simulating-a-trusted-monitor-r682): 3rd place by peer review, AI Control Hackathon 2025. We offer initial investigations into whether the untrusted model can 'simulate' the trusted monitor: is U able to successfully guess what… - [Adding Document Summaries to Control Arena](https://apartresearch.com/sprints/projects/adding-document-summaries-to-control-arena-fggr): 4th place by peer review, AI Control Hackathon 2025. One of the current focuses of AI Control research is finding new settings within which to examine the control problem. In this hackathon,… - [Can models use their Chain-of-Thought to attack overseers?](https://apartresearch.com/sprints/projects/can-models-use-their-chainofthought-to-attack-overseers-prcv): 1st place by peer review, AI Control Hackathon 2025. This project explores vulnerabilities in AI evaluation mechanisms, specifically focusing on how AI agents might influence their overseeing… - [Schelling coordination via agentic loops](https://apartresearch.com/sprints/projects/schelling-coordination-via-agentic-loops-azvd): 2nd place by peer review, AI Control Hackathon 2025. Capability to collude undermines untrusted monitoring protocols. We propose a novel way to evaluate acausal coordination capabilities of… - [Detecting Malicious AI Agents Through Simulated Interactions](https://apartresearch.com/sprints/projects/detecting-malicious-ai-agents-through-simulated-interactions): Social Sciences Track Prize, Women in AI Safety Hackathon. This research investigates malicious AI Assistants’ manipulative traits and whether the behaviours of malicious AI Assistants can be… - [Red-teaming with Mech-Interpretability](https://apartresearch.com/sprints/projects/red-teaming-with-mech-interpretability): Mechanistic Interpretability Prize, Women in AI Safety Hackathon. Red teaming large language models (LLMs) is crucial for identifying vulnerabilities before deployment, yet systematically creating… - [Morph: AI Safety Education Adaptable to (Almost) Anyone](https://apartresearch.com/sprints/projects/morph-ai-safety-education-adaptable-to-%28almost%29-anyone): Education Track Prize, Women in AI Safety Hackathon. One-liner: Morph is the ultimate operation stack for AI safety education—combining dynamic localization, policy simulations, and ecosystem… - [AntiMidas: Building Commercially-Viable Agents for Alignment Dataset Generation](https://apartresearch.com/sprints/projects/antimidas-building-commercially-viable-agents-for-alignment-dataset-generation): 1st place by peer review, AI Safety Entrepreneurship Hackathon. AI alignment lacks high-quality, real-world preference data needed to align agentic superintel- ligent systems. Our technical innovation… - [AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control](https://apartresearch.com/sprints/projects/autosteer-weight-preserving-reinforcement-learning-for-interpretable-model-control): 1st place by peer review, Reprogramming AI Models Hackathon. Traditional fine-tuning methods for language models, while effective, often disrupt internal model features that could provide valuable… - [Promoting School-Level Accountability for the Responsible Deployment of AI and Related Systems in K-12 Education: Mitigating Bias and Increasing Transparency](https://apartresearch.com/sprints/projects/promoting-school-level-accountability-for-the-responsible-deployment-of-ai-and-related-systems-in-k-12-education-mitigating-bias-and-increasing-transparency): 1st place by peer review, Howard University AI Safety Summit & Policy Hackathon. This policy memorandum draws attention to the potential for bias and opaqueness in intelligent systems utilized in K–12 education, which… - [Robust Machine Unlearning for Dangerous Capabilities](https://apartresearch.com/sprints/projects/robust-machine-unlearning-for-dangerous-capabilities): 1st place by peer review, AI Policy Hackathon at Johns Hopkins University. We test different unlearning methods to make models more robust against exploitation by malicious actors for the creation of bioweapons. - [Diamonds are Not All You Need](https://apartresearch.com/sprints/projects/diamonds-are-not-all-you-need): 1st place by peer review, Agent Security Hackathon. This project tests an AI agent in a straightforward alignment problem. The agent is given creative freedom within a Minecraft world and is… - [DarkForest - Defending the Authentic and Humane Web](https://apartresearch.com/sprints/projects/darkforest-defending-the-authentic-and-humane-web): 1st place by peer review, Hackathon for Technical AI Safety Startups. DarkForest is a pioneering Human Content Verification System (HCVS) designed to safeguard the authenticity of online spaces in the face of… - [AI Alignment Knowledge Graph](https://apartresearch.com/sprints/projects/ai-alignment-knowledge-graph): 1st place by peer review, Research Augmentation Hackathon. We present a web based interactive knowledge graph with concise topical summaries in the field of AI alignement - [Sandbag Detection through Model Degradation](https://apartresearch.com/sprints/projects/sandbag-detection-through-model-degradation): 1st place by peer review, Deception Detection Hackathon: Preventing AI deception. We propose a novel technique to detect sandbagging in LLMs by adding varying amount of noise to model weights and monitoring performance.