Our research focuses on critical themes in AI safety, including interpretability methods to understand how AI systems make decisions, process supervision techniques to ensure models behave as intended throughout their development and use, and benchmark tasks to evaluate safety and alignment with human values. We aim to produce foundational research enabling the safe and beneficial development of advanced AI.

Sleeper Agents

Published

for

with

Anthropic

 on 

January 14, 2024

Sleeper Agents

Published

for

with

Anthropic

 on 

January 14, 2024

Neuroplasticity in LLMs

Published

for

with

 on 

January 3, 2024

Neuroplasticity in LLMs

Published

for

with

 on 

January 3, 2024

DeepDecipher

Published

for

with

NeurIPS 2023 XAI in Action Workshop

 on 

November 28, 2023

DeepDecipher

Published

for

with

NeurIPS 2023 XAI in Action Workshop

 on 

November 28, 2023

Mechanistic Interpretability

Safety Benchmarks

Interpreting Reward Models in RLHF-Tuned Language Models Using Sparse Autoencoders

Luke Marks, Amir Abdullah, Luna Mendez, Rauno Arike, Philip Torr, Fazl Barez

arXiv

[Read Paper}

Published on

November 28, 2023

at

arXiv

Neuron to Graph: Interpreting Language Model Neurons at Scale

Alex Foote¹*, Neel Nanda², Esben Kran¹, Ionnis Konstas³, Shay Cohen⁴, Fazl Barez^1,4,5*

RTML workshop at ICLR 2023

[Read Paper}

Published on

May 5, 2023

at

RTML workshop at ICLR 2023

DeepDecipher

NeurIPS 2023 XAI in Action Workshop

[Read Paper}

Published on

November 28, 2023

at

NeurIPS 2023 XAI in Action Workshop

Understanding Addition in Transformers

Philip Quirke, Fazl Barez

arXiv

[Read Paper}

Published on

November 25, 2023

at

arXiv

Locating Cross-Task Sequence Continuation Circuits in Transformers

Michael Lan, Fazl Barez

arXiv

[Read Paper}

Published on

November 7, 2023

at

arXiv

Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark

Jason Hoelscher-Obermaier¹*, Julia Persson¹*, Esben Kran¹, Ionnis Konstas², Fazl Barez^1,3*

ACL 2023

[Read Paper}

Published on

July 10, 2023

at

ACL 2023

The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python

Antonio Valerio Miceli-Barone¹*, Fazl Barez¹*, Ioannis Konstas², Shay B. Cohen¹

ACL 2023

[Read Paper}

Published on

May 24, 2023

at

ACL 2023