The Control Hackathon brings together researchers, engineers, security professionals, and AI enthusiasts to tackle the emerging challenges in AI control - techniques that mitigate security risks from AI even when the AI itself might be trying to subvert them.

🔒 About the Hackathon

As AI systems become more capable and autonomous, ensuring robust control mechanisms becomes increasingly critical. This hackathon, co-organized by Redwood Research, and the UK AI Safety Institute, provides a practical extension to ControlConf (March 27-28), allowing participants to apply theoretical concepts to real-world control challenges.

💡 Challenge Tracks

Note: this list is not exhaustive and we will be posting a list of Open Problems in Control from Redwood Research here soon

1. ControlArena Challenges

Work with the ControlArena from UK AISI to develop, test, and evaluate control protocols in realistic deployment environments. Try to design robust protocols that can withstand adversarial agents or develop novel attack strategies that break existing protocols.

2. Control Protocol Design

Design and implement novel control protocols that effectively restrict AI systems from performing harmful actions while maintaining their usefulness for legitimate tasks. Focus areas include:

• Runtime monitoring systems

• Adversarial stress testing

• Formal verification approaches

• Bounded optimization techniques

3. Red Teaming & Vulnerability Research

Design and implement strategies to "red team" AI systems, attempting to subvert safety mechanisms while adhering to ethical guidelines.

Focus areas include:

• Generating adversarial inputs to bypass monitoring systems.

• Testing robustness against intentional misalignment or malicious behavior

‍

👥 Who Should Participate

We welcome participants from diverse backgrounds, including:

• Participants from ControlConf joining in person at LISA. Luma event info will follow up.

• AI researchers and engineers

• Information security professionals

• ML and systems engineers

• Students in related fields

• Policy researchers interested in technical AI safety

No prior experience with AI control specifically is required, though familiarity with machine learning, programming, or information security is helpful.

‍