While much AI safety research focuses on large language models, the AI systems being deployed in the real world are far more complex. Enter the realm of Agents — sophisticated combinations of language models and other programs that are reshaping our digital world.
During this hackathon, we'll put our research skills to the test by diving deep into the world of agents:
Join us to uncover the unknowns of agent security by signing up below!
The development of AI has brought about systems capable of increasingly autonomous operation. AI agents, which integrate large language models with other programs, represent a significant step in this evolution. These agents can make decisions, execute tasks, and interact with their environment in ways that surpass traditional AI systems.
This progression, while promising, introduces new challenges in ensuring the safety and security of AI systems. The complexity of agents necessitates a reevaluation of existing safety frameworks and the development of novel approaches to security. Agent security research is crucial because it:
During this hackathon, you'll have the opportunity to:
You will join in teams to submit a report and code repository of your research from this weekend. Established researchers will judge your submission and provide reviews following the hackathon.
Doroteya Stoyanova, Computer Vision Intern
I learnt so much about AI Safety and Computation Mechanics. It is a field I never heard of, and it combines two of my interests - AI, and Physics. Through the hackathons I gained valuable connections, learnt a lot by researchers, people with a lot of experience and this will help me in my research-oriented career-path.
Kevin Vegda, AI Engineer
I loved taking part in the AI Risk Demo-Jam by Apart Research and LISA. It was my first hackathon ever. I greatly appreciate the ability of the environment to churn out ideas as well as to incentivise you to make demo-able projects that are always good for your CV. Moreover, meeting people from the field gave me an opportunity to network and maybe that will help me with my career.
Mustafa Yasir, The Alan Turing Institute
[The technical AI safety startups hackathon] completely changed my idea of what working on 'AI Safety' means, especially from a for-profit entrepreneurial perspective. I went in with very little idea of how a startup can be a means to tackle AI Safety and left with incredibly exciting ideas to work on. This is the first hackathon in which I've kept thinking about my idea, even after the hackathon ended.
You have a unique chance to win during this hackathon! With our expert panel of judges, we'll review your submissions on the following criteria:
This hackathon is for anyone who is passionate about AI safety and secure systems research. Whether you're an AI researcher, developer, entrepreneur, or simply someone with a great idea, we invite you to be part of this ambitious journey. Together, we can build the tools and research needed to ensure that agents develop safely.
By participating in the Agent Security Hackathon, you'll:
If [AI] agents advance to a level of intelligence surpassing human capabilities and develop ambitions, they could potentially attempt to seize control of the world, resulting in irreversible consequences for humanity.
- "The Rise and Potential of LLM-Based Agents"
AI agents are "robots in cyberspace" (He et al. 2024), systems with a brain that orchestrates actions from perception.
As we explore agent safety during this hackathon, our work will ensure that the world is safe from high-risk agent deployments and one of the main risks to avoid is the possibility that AI agents "go rogue" (Bengio 2023, He et al. 2024) - that they take autonomous actions outside the oversight of humans and cause catastrophic damage or otherwise disenfranchise society.
Required reading:
Optional reading:
There's a few ways to approach this problem: 1) we implement algorithms that make agents verifiably safe or 2) we evaluate when an agent is worrying us and build automatic capabilities to shut it down. Let's make a few ideas for each of these categories:
With "building safer agents" and "improving the deployment infrastructure to support control and safety" in mind:
With "detecting and monitoring catastrophic risk from agents" in mind:
Let's make agents safe.
The schedule runs from 4 PM UTC Friday to 3 AM Monday. We start with an introductory talk and end the event during the following week with an awards ceremony. Join the public ICal here.You will also find Explorer events, such as collaborative brainstorming and team match-making before the hackathon begins on Discord and in the calendar.
If your're in Santiago and want to join us message @weibac on Telegram
If your're in Santiago and want to join us message @weibac on Telegram
We look forward to welcoming you at the EA Hotel: York Street 36, Blackpool, UK. Here, you will find a cozy bed, good food, and a merry little community of aspiring effective altruists.
We look forward to welcoming you at the EA Hotel: York Street 36, Blackpool, UK. Here, you will find a cozy bed, good food, and a merry little community of aspiring effective altruists.
Join us for a remote location of the next Apart Research Hackathon on Agent Security at the LEAH office in Farringdon, London.
Join us for a remote location of the next Apart Research Hackathon on Agent Security at the LEAH office in Farringdon, London.
Av. Unión #163 Piso 1 Col. Lafayette Guadalajara, Jalisco, México 44140
Av. Unión #163 Piso 1 Col. Lafayette Guadalajara, Jalisco, México 44140
Location will be a coworking space in Hanoi that can accommodate up to 30 people. Exact location is NovaUp 22nd Thành Công St., Thành Công, Ba Đình, Hà Nội, Vietnam. More details are available in the luma link.
Location will be a coworking space in Hanoi that can accommodate up to 30 people. Exact location is NovaUp 22nd Thành Công St., Thành Công, Ba Đình, Hà Nội, Vietnam. More details are available in the luma link.
Join us for a weekend hackathon at Skybox 1 and 2 in DTU Skylab . Whether you're new to the topic or have some experience, this hackathon is a great opportunity to get hands-on experience with the security of agent-based systems.
Join us for a weekend hackathon at Skybox 1 and 2 in DTU Skylab . Whether you're new to the topic or have some experience, this hackathon is a great opportunity to get hands-on experience with the security of agent-based systems.
The report should include:
Submissions will be judged based on the following criteria:
Here are the entries for the Agent Security Hackathon 2024