This Challenge has been concluded

Over the July 25th-27th weekend intensive, participants tackled critical challenges by applying physics methodologies—from thermodynamics and statistical mechanics to computational mechanics—to understand and control AI systems. The event featured both Project and Exploration tracks, encouraging participants to bridge theoretical physics insights with practical AI safety solutions through collaborative innovation.
Following are the prize winners:

1. First Prize: Constrained Belief Updates in Transformer Representations:

   Brianna Grado-White

   This work extended computational mechanics theory to analyze how transformers implement optimal prediction under constraints. It showed belief state geometry across layers and that attention patterns decay exponentially with distance. Despite complex challenges, it bridged physics-inspired theory and practical interpretability, helping understand AI reasoning.

2. Second Prize: Momentum-Point-Perplexity Mechanics in Large Language Models

   by Lorenzo Tomaz, Thomas Jones
   This work found that energy remains approximately conserved across 20 different models. Their theoretical insights led to Jacobian steering—a control method that improves output quality while respecting the underlying physics, offering both fundamental understanding and practical applications for AI safety.

3. Third Prize: Local Learning Coefficients in Reinforcement Learning:
   by Jeremias Ferrao, Ilija Lichkovski

   This project applies Singular Learning Theory to reinforcement learning, demonstrating how Local Learning Coefficients (LLCs) detect training phase transitions. By tracking reward components directly rather than derived metrics, LLC spikes correlate with capability milestones. This scalable approach enhances the understanding of how AI systems learn complex behaviors, with significant safety implications.

4. AI agentic system epidemiology

   by Valentina Schastlivaia, Aray Karjauv
   Using Physics-Informed Neural Networks to solve the governing equations, they created a framework for monitoring AI system health and predicting intervention effectiveness—providing a novel lens for understanding systemic AI risks.

Please find the link to the presentation here: https://docs.google.com/presentation/d/1IWKW9gMtc2qjfQAdWlRcT0EnJzeuJX3h6XOZC2oPOT4/edit?usp=sharing

Uniting Physics to Solve AI Safety

Join us for this AI Safety x Physics Grand Challenge to use your valuable skillset on chosen problem spaces for physicists!

You'll work alongside peers to create an original research project during this two day sprint with the guidance of expert mentors in the unique intersection of physics and AI safety. Be ready for excitement, progress, and, last but not least, prizes!

Physics and AI Safety?

The complexity of neural networks, the topology of their latent space, and the dynamics of the systems that deploy AGI today is an ideal topic of study for any physicist. We are at the absolute frontier of understanding learning dynamics through fundamental physics and during this event, you will become an integral part of this field's future!

Research Tracks

Employ your ML / AI experience from interpretability, benchmarking, or training to develop an original project employing the novel tools that physicists have introduced to the field in the past years.

Examples include the projects we've defined, such as operationalizing the natural abstraction hypothesis for empirical tests of compression or predictability of learned abstractions.

Propose novel research agendas to tackle the most pressing AI challenges or explore a potential bridge between an AI safety problem and a physics concept.

An example could be a proposal, outlining a novel physics-based solution to a key AI safety problem, with supporting material drawn from prior work. 

Join

And experience…

• Expert mentorship from leaders in the field of physics and AI safety

• Structured problem areas with starter materials and concrete research directions

• Global community of highly talented participants who are ambitiously solving top challenges

• Follow-up support for exceptional projects through the Apart Research programs

• Optional pre-event orientation and brainstorming for those new to AI safety

Shape the Future!

Join us in applying your academic training to one of the most important technical challenges of our time. Connect with a growing community of researchers safeguarding AI systems for the benefit of humanity.

FAQ

What Background Knowledge Is Required?

This challenge welcomes participants with physics backgrounds who are interested in applying rigorous methodologies to AI safety. While familiarity with machine learning is helpful, we encourage participation from:

• Physics PhD students, postdocs, and faculty across all subfields

• Theoretical physicists interested in complex systems and statistical mechanics

• Experimental physicists with experience in data analysis and modeling

• Computational physicists familiar with simulation and numerical methods

• Physicists working at the intersection of physics and computer science

• Anyone passionate about applying physics thinking to beneficial AI development

How Are Teams Formed?

Teams will self-organize during the first day through structured activities and interest-matching sessions. You can also form teams in advance with other registered participants via our Discord channel: https://ais.pub/disc

Teams should ideally combine:

• Different physics backgrounds (theory, experiment, computation)

• Varied experience levels with AI/ML

• Complementary skills in mathematics, programming, and research

What Are The Expected Outputs?

Teams should produce research outputs and reports that demonstrate progress toward applying physics methodologies to AI safety challenges. The most promising projects may evolve into:

• Academic papers or technical reports

• Continued research collaboration with physics-AI safety researchers

• Integration into ongoing research programs at leading institutions

• Follow-up funding opportunities for extended research

• Theoretical frameworks for future empirical validation

Will Compute Resources Be Provided?

Yes! Lambda is providing $400 in computing credits to each participating team, giving you access to powerful cloud instances (including A100s) to bring your ideas to life.

What Challenge Track Should I Choose?

You must align your project with one of our two tracks:

Project Track: This is a typical hackathon submission based on the starter materials or an original idea. We expect most of these to provide incremental theoretical progress and/or empirical evidence for (or against) an existing idea.

Relevant for higher context participants (i.e. those with experience running experiments on NNs or with an existing physics-AI agenda they want to accelerate progress on).

Exploration Track:  Part of the idea for this hackathon is to open up and test the limits of a ‘physics for AI safety’ opportunity space. We have thought of some, but likely many more that would be exciting. Instead of traditional ‘hacking’, this will be a written submission, for example: 

1. A proposal, outlining a novel physics-based solution to a key AI safety problem, with supporting material drawn from prior work. 

2. A distillation or literature review bridging a CS/ML/AI idea with one from physics, with clear reference to an AI safety problem area. 

We think this track has the potential to brainstorm some truly innovative ideas, while also allowing lower-context participants to get involved. For example, a ‘papers’ participant or group could partner with a ‘project’ group, resulting in a small research gain and a better understanding of the bigger picture. 

What Problem Area Should I Focus On?

Your project should address one or more of our five problem areas:

Problem Area 1: Theory/Practice Gap for AI Interpretability - Apply physics methodologies (renormalization, computational mechanics, statistical mechanics) to build theoretical foundations for mechanistic interpretability

Problem Area 2: Learning, Inductive Biases, and Generalization - We want to understand how architecture, initialization, data distribution, and training procedures give rise to inductive biases—and in turn, how these biases control what is learned, when generalization occurs, and why models may fail.

Problem Area 3: Mathematical Models of Data Structure - AI alignment faces the challenge of understanding an AI's internal world model to prevent harmful actions. Physics methods, like coarse-graining and renormalization, could provide mathematical models to describe AI's learned representations and improve our understanding of AI's abstraction, generalization, and world modeling.

Problem Area 4: Quantitative Bounds on AI Behavior - This challenge focuses on using physics-inspired uncertainty quantification to establish quantitative bounds on AI behavior, ensuring alignment even when systems develop unanticipated strategies. Your work is crucial for guaranteeing ethical and reliable AI.

Problem Area 5: Scaling Laws, Phase Transitions, and Emergence - Uncover the physical foundations of AI performance, universal limits, and how smooth scaling reconciles with emergent capabilities.