Dear Apart Community,

Welcome to our newsletter - Apart News!

At Apart Research there is so much brilliant research, great events, and countless community updates to share.

This week's edition of Apart News contains the inaugural edition of our Researcher Spotlights, introduces a new Apart Team member, highlights more of our paper and workshop successes, and reveals a new international Jam Site.

New Paper on Vision-Language Models

Our Research Assistant, Clement Neo, has a new paper out called Towards Interpreting Visual Information Processing in Vision-Language Models. Studying Vision-Language Models (VLMs) like LLaVA to understand how they process objects in images. Co-authored with Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, Fazl Barez.

They find surprising insights about how these models identify objects in images and how their inner representations develop through the layers. Congratulations to the team.

New Apart Team Member

Jaime is our new Research Acceleration Engineer: he has a really important combination of software engineering skills and a deep understanding of machine learning and AI research processes. As our Research Acceleration Engineer, he will play a crucial part in enabling the research team to work more effectively and efficiently, ultimately contributing to Apart's mission. So welcome to the team, Jaime!

Workshop Successes

New papers have been successfully admitted to various ML workshops this week, including: Sandbag Detection through Model Impairment which has been accepted the SATA & SoLaR workshops, Modelling the oversight of automated interpretability against deceptive agents on sparse autoencoders to SATA, and CryptoFormalEval: Integrating Large Language Models and Formal Verification for Automated Cryptographic Protocol Vulnerability Detection which was accepted to the System2 NeurIPS workshop.

Nobel Prizes

Congratulations to Demis Hassabis and Google DeepMind (alongside David Baker & John M. Jumper) for receiving the Nobel Prize in Chemistry. And also to Prof. Geoffrey Hinton for receiving his Nobel Prize in Physics. Earlier this year, Hinton reminded us that: "We're entering a period of great uncertainty where we're dealing with things [capable AI] we've never dealt with before [...] we can't afford to get it wrong with these things."

It is clear that as AI becomes increasingly capable at solving open problems in the sciences it can bring about real benefits to the world. Esben wrote in a tweet that it is: "Amazing to see @demishassabis on the list for the chemistry Nobel prize! Very well deserved. Definitely not on my bingo card that two AI safety advocates would win Nobel prizes this year but it makes a lot of sense."

Connor tweeted that: "The new @NobelPrize Laureate in Physics for his pioneering work in the development of AI, Prof. Geoffrey Hinton has warned us repeatedly: we need to ensure model capabilities do not pose a risk to humanity. He believes AI on its current trajectory may disenfranchise humanity."

Researcher Spotlight: Akash Kundu

This is the first of our Researcher Spotlights: which will shine a light on the global community behind Apart Research. This first edition focuses on Akash Kundu, a Lab Fellow from India.

Tell us a bit about yourself, Akash!

"Hi, I am Akash Kundu, a pre-final year CS undergrad from Kolkata, India. I have had the opportunity to work at Apart Research as a research fellow working on refining my research sprint ideas to research papers. When I'm not working you'll find me singing, beatboxing, and playing the guitar."

How did you first hear about Apart Research?

"I first heard about Apart in 2023 through my friend Jord, who’s also an Apart Fellow. He asked me if I’d like to join him for the model evaluations hackathon, organized with Apollo Research. Jason [Apart's Co-Director] helped guide us as a mentor during the hackathon, providing support to keep us on track. We ended up finishing as second runners-up and were invited to join the Apart Lab as fellows to refine our idea into a conference research paper. Since then, I’ve participated in several sprints and have been working with Apart ever since."

How many Sprints have you done now?

"To be honest, I’ve lost count! Since November 2023, I’ve registered for almost 80% of the sprints that took place. I really enjoy spending the weekends hacking away and building cool projects. My most recent work was during the Concordia Hackathon where our "cooperative" agent finished first on the final leaderboard.

"I’ve had a great time participating in these sprints, meeting new teammates who’ve become meaningful connections. The office hours gave us guidance when we got stuck and helping us steer our ideas in the right direction. I highly recommend participating in these sprints; they’re a great way to learn, meet new people, and get feedback from experts on your ideas."

And now you've been writing academic papers?​

"Yes! Thanks to Apart, I’ve been involved in writing a few research papers. The cross-lingual project from November evolved into a study where we examined why low-resource languages tend to jailbreak LLMs, and we worked on predicting the causality of these jailbreaks. Another ongoing project is based on the Dark Patterns work."

Is your future plan to stay in AI Safety?

"I’m still figuring things out! I plan to get involved with impactful Alignment research and aim to significantly contribute to aligning AI systems. I’m not entirely sure what the right path forward is yet—I might go for a PhD or continue researching independently on important problems.

"I’m confident that Apart would support me whenever needed. Right now, I’m trying to get into research programs like MATS and LASR Labs. I’m pretty flexible about my future plans and open to finding the right path as I go along."

New Jam Site

We also have a brand new Jam Site in Bangalore, India. One of its coordinators, Aditya, said: "I hope people in Bangalore can use this to find out about more events happening." Great work team!

Opportunities

• Sign up to our next AI Policy Hackathon at Johns Hopkins University, happening October 26-27, 2024 in Washington DC! Join us for a weekend of collaboration, problem-solving, and networking as you work with like-minded peers to tackle real-world policy challenges related to AI.
• ​AI Institute Fellowship Program.
• ​Program Scientist, AI and Advanced Computing Institute.

Have a great week and let’s keep working towards safe AI.

‘We are an AI safety lab - our mission is to ensure AI systems are safe and beneficial.’