.svg){kind=icon}
Researcher Spotlights highlight the global community at the heart of Apart Research. We have so many incredible, talented researchers at Apart Research - this series will help you to get to know them a little better. Jacob Haimes is our Research Assistant and used to be a Fellow with us before that. Here is his AI Safety story.
01. Tell us a little about where you're from?
"Hi, my name is Jacob Haimes and I am from Boulder, Colorado, which is right at the foothills of the Rocky Mountains. I grew up here in Boulder and when I was applying for colleges, I only applied to one because I knew I wanted to go to CU Boulder."
"So I went there for undergrad in mechanical engineering and then I did my master's in computational modeling, also through the mechanical engineering department, but focused on optimization and I had a minor in computer science as well."
02. When did you get interested in AI safety?
"So about half a year or so after starting my first job out of college, I started thinking basically like, this is okay. It was healthcare information systems. Yeah. And I liked that I was doing positive impact, but I felt like I could be using my technical skills more and that I could be potentially doing more, I guess, as well."
"And so I did a retrospective or like a reflection. That's what I was looking for. Reflection on what I wanted to do and what mattered to me. [...] I felt that I could be doing a lot more for my community. "
"AI was taking off quickly, but was still nascent, and if I started working on these things now, I could have a potentially like very outsized impact as defining the paradigms that exist within a field as it's being created is probably going to have pretty long-term effects."
03. When did you first hear about Apart Research?
"So this would have been in around November of 2023 when I first heard about Apart and I was just attending some talks at an EAGx virtual and Esben who's one of the co-founders of Apart was the one who started working on this project."
"And he was giving a talk on, I think it was actually called 'Let's Get Into AI Safety', which at that point was already the name of my podcast. So I attended that and he signposted that a hackathon was happening in the next week, it was on evaluations and I basically thought like this seems like an opportunity. This is the community that I'm interested in."
"This is in the area that I'm more interested in, especially at that point. And I mean, we both chose basically the same name, so that's gotta mean something, and that's how I first learned about Apart."
04. Tell us about your Fellowship with Apart?
"Yeah. So after that, so the hackathon happened. Um, and I actually recorded the episode or I recorded the, the conversations that our group had and I, uh, two of them are up on the Into AI Safety podcast. And, uh, I plan on eventually releasing the others, uh, but it's just been a lot of work to get things up and, and keep maintaining that as well. So that they will come out eventually. But we were invited to join the APART fellowship and. Uh, two of the four of us decided to join, um, the other two wanted to focus on some other things, uh, which is totally, totally valid."
"And basically we then took our hackathon idea, which was already, I would say pretty well defined in terms of what we wanted to do, and applied for, uh, at least one, maybe two grants, um, and started turning it into more of like, a real research proposal, um, and understanding how we could make it into a conference paper, um, at, uh, that was actually accepted."
05. What is Apart Lab Studio?
"Yeah, so this is something that we're really excited about. We spent a long time thinking about the best way to take a project from hackathon to end of hackathon. You've just done some great research in a very short amount of time, and now you have something, but it isn't necessarily in a state that is the best for sharing."
"And we think the best way to increase impact there, both for like participants and just overall, is to start with a quick push to sort of polish that up, add maybe a couple extra things like an uncertainty analysis, or some additional like margin of error, or additional models that you're running this test on or something like that, and then make it look pretty and share that..."
"The next step for one project isn't the same as it is for another project and figuring out what that is and what will be most effective and also how to best share that information uh is a lot of what this I guess like approximately Two weeks is uh, with the end being publishing a blog post..."
06. Tell us about your Apart paper: ‘Benchmark Inflation?’
"Okay um yeah so so the end paper was called uh or is called benchmark inflation revealing llm performance gaps using retro hold outs and essentially in this the question we're asking is we know that um evaluation gaming in the form Of data leakage, so the test data getting into the training data of LMS benchmarks having their questions and corresponding answers be put directly into the training sets of LLMS."
"We know that's happening at this point because there's tons of literature on this, but what we don't know is... what is the concrete impact... what we found doing uh this benchmark inflation is that that's kind of like a that's not the case. Instead, it does have a meaningful impact."
07. Tell us about an important AI breakthrough in 2024?
"Okay, this is actually a relatively new paper, I believe, that has, I think will have pretty big implications. But basically, it's finding say that like quantization, so being able to make these big models smaller, without a lot of loss, is coming to an end. And that would imply that scaling is going to get a lot harder, and more costly. And so that, that may indicate an end, or at least a plateau, or stall in this idea of just like, pour more compute into the pre-training of models."
08. If you a budding researcher, how do you get into the field to make sure the next few years of rapid AI progress goes well?
"At this point, a lot of the AI safety, or just even AI and ML research is, is extremely competitive. And so it's like, first off, if you aren't in the Bay Area, DC, or London already, and you're not willing to move there, then like, good luck. In addition, it just like is really competitive. So even if you are willing to move there, or you are already there, it may be really difficult to find and secure like actual sustainable opportunities... The solution that I found and went to, at this stage, like for me, was, okay, there, there aren't any opportunities that are just going to fall into my lap. If I want to do this, I need to make my own opportunities."
09. Any hobbies?
"Well, things have gotten a little bit more, uh, busy recently because I, I am working with Apart and Kairos FM with that, which I'm hosting two podcasts. And then I also do some work with the Odyssean Institute, um, on, on some more governance, uh, democratic process, uh, research, but bringing that to AI and, and AI governance..."
"I love role-playing games. Um, so sorry, uh, the person interviewing me just did a very weird hand gesture at me... I love role-playing games, so things like Dungeons and Dragons, but also, uh, other ones like Blades in the Dark, uh, Grimwild, um, anything really, uh, I do a lot of DMing as well. So some people might consider that to be less, uh, like more work after I'm done with work, but I, I think it's just so, so fun."
* * *
[Relevant links]:
https://arxiv.org/abs/2411.04330
https://kairos.fm/muckraikers/
https://kairos.fm/intoaisafety/
https://kairos.fm/posts/research/benchmark-inflation/
https://mastodon.social/@kairosfm
https://bsky.app/profile/kairosfm.bsky.social
https://www.youtube.com/@KairosFMNetwork
