A tool to map the sprawling landscape of AI alignment research

This week we have been in San Francisco for our Apart Retreat, where we attended conferences, saw old friends, and visited other AI labs to talk about frontier AI.

This week, we celebrate ICLR conference oral awards for two of our papers, launch our Women in AI Safety hackathon, and more.

Apart Research developed DarkBench to uncover dark patterns - application design practices that manipulate a user’s behavior against their intention - in some of the world's most popular in LLMs.

We are happy to share the significant progress made by the first batch of Apart Research's Studio projects.

This week Esben gave a talk in Paris and our inaugural Studio Progress Report is released soon.

This week some of the team are in Paris & we have just published an Apart Lab Studio research blog about catching AI hackers.

In his Hackathon Round-Up we check out the winners of our AI Entrepreneurship Hackathon.

This week we reveal our AI Startup Hackathon winners and have a look at the Apart Lab paper just accepted to ICLR's 2025 conference.

Myra reveals how Goodfire's groundbreaking API enabled 200+ researchers at Apart's global hackathon to advance AI interpretability, demonstrating new ways to make AI systems more transparent and controllable.

Goodfire's Myra reveals how their groundbreaking API enabled 200+ researchers at Apart's global hackathon to advance AI interpretability, demonstrating new ways to make AI systems more transparent and controllable.

Apart Research's newest research achieves promising results from Latent Adversarial Training.

In this week's Apart News we look over promising new LAT research and get a Hackathon insider's account from Archana.

Johns Hopkins University hosted its first AI Policy Hackathon in partnership with us at Apart Research. Here's what participants and organizers had to say about bridging the gap between technology and policy.

2024 was the biggest and most impactful year of Apart Research so far.

This Apart Lab Studio research blog attempts to ascertain the current state of AI-powered hacking in the wild through an innovative 'honeypot' system designed to detect LLM-based attackers.

In this week's Apart News we preview some of the Hackathons we are most excited for in 2025.

Our Hackathon round-up showcases our global sprints community.

Apart Research's newest paper looks at LLM-assisted benchmark analysis.

In this week's Apart News we are at NeurIPS in Canada.

In this week's Apart News we have a *brand new* video edition of our Researcher Spotlight series.

In this week's Apart News we invite you to revisit Apart Research's incredible ​2024​ with us.

In this week's edition of Apart News we ask just how impactful a donation is to Apart Research and take a look at the ability of LLMs to predict neuroscience results.

Apart News is our newsletter to keep you up-to-date.

Apart Research's newest paper offers a systematic way to evaluate how well Large Language Models (LLMs) can identify vulnerabilities in cryptographic protocols.

Co-Director Esben gives us his thoughts on how impactful donating to Apart Research is.

Apart News is our newsletter to keep you up-to-date.

Our new Apart Lab Studio is designed to bridge the gap between weekend hackathon projects and a fully-fledged AI Safety research career.

Apart News is our newsletter to keep you up-to-date.

Our Researcher Spotlight series highlights the global community at the heart of Apart Research.

Apart News is our newsletter to keep you up-to-date.

Esben Kran gives us some of his thoughts on ideas relevant to AI safety, decision-making, and more.

Apart Research's newest paper, Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities, creates a novel cyber offense capability benchmark.

Our Hackathon round-up showcases our global 'sprints' community.

Apart News is our newsletter to keep you up-to-date.

Apart News is our newsletter to keep you up-to-date.

Apart Research's newest paper finds that many public benchmarks may no longer provide accurate evaluations due to the inclusion of test data in training datasets.

Our Researcher Spotlight series highlights the global community at the heart of Apart Research.

Our Researcher Spotlight series highlights the global community at the heart of Apart Research.

Apart News is our newsletter to keep you up-to-date.

Agent safety research is difficult because it involves many different types of entities and wide range of vulnerabilities and failure modes. As a result, it’s hard to develop research that generalizes to all agents. However, we need to give it a shot!

Apart News is our newsletter to keep you up-to-date.

Apart News is our newsletter to keep you up-to-date.

Apart Research's newest paper (alongside academics from the University of Oxford, Cambridge, and Cynch.ai) looks at whether models actually internalize human preferences or not. But why does this matter?

Apart News is our newsletter to keep you up-to-date.

Apart News is our newsletter to keep you up-to-date.

This post details the top projects from our technical AI safety startups hackathon where researchers and entrepreneurs joined from across the world.

Read about how Big Tech's AI race leaves safety in the dust, non-profits struggle to keep up, and the challenges for-profit AI safety ventures must overcome to leverage resources and make a real impact.

This June, Apart Research and Apollo Research joined forces to host the Deception Detection Hackathon, bringing together students, researchers and engineers from around the world to tackle one of the most pressing challenges in AI safety: Preventing AI from deceiving humans.

Our 128 participants submitted more than 200 project ideas, 100 detailed task specifications, and more than 20 complete implementations! In this post, we also get an exclusive interview with one of the winners.

Research hackathons are an amazing way to dive into a new field, collaborate with passionate people, and create impactful projects in just a short weekend.

Participate online or in-person on the weekend 3rd to 5th May in an exciting and intense AI safety research hackathon focused on demonstrating and extrapolating risks to democracy from real-life threat models.

In this collaboration between METR and Apart, you get the chance to contribute directly to model evaluations research.

Organizing a hackathon can bring a unique and exciting energy to people interested in AI safety research! This post summarizes how you can organize a successful hackathon.

Our Researcher Spotlight series highlights the global community at the heart of Apart Research.

AI development attracts more than $67 billion in yearly investments, contrasting sharply with the $250 million allocated to AI safety. This gap suggests there's a large opportunity for AI safety to tap into the commercial market. The big question then is, how do you close that gap?

With the research hackathon, your journey into the world of AI safety is definitely not over! Besides the chance to join the Apart Lab Fellowship, we have collected a bunch of resources here for you to dive even deeper into the field!

There are many reasons to run a hackathon but some of the main ones are that hackathons are an amazing way to engage the local groups in AI security research and create a sense of community.

Written by Neel Nanda, who previously worked on mech interp under Chris Olah at Anthropic, who is currently a researcher on the DeepMind mechanistic interpretability team.

Check out the top projects from the "Scale Oversight" hackathon hosted in February 2023: Playing games with LLMs, scaling of prompt specificity, and more.

See the winning projects from the AI testing hackathon held in December 2022: Trojan networks, unsupervised latent knowledge representation, and token loss trajectories to target interpretability methods.

See winning projects from the language model hackathon hosted November 2022: GPT-3 shows sycophancy, OpenAI's flagging is biased, and truthfulness is sensitive to prompt design.

Read the winning projects from the interpretability hackathon hosted in November 2022: Automatic interpretability, backup backup name mover heads, and "loud facts" in memory editing.