Keep Apart Research Going: Donate Today
Jul 1, 2024
Werewolf Benchmark
Luhan Mikaelson, Zach Nguyen, Andy Liu, Jord Nguyen, Akash Kundu
Summary
In this work we put forward a benchmark to quantitatively measure the level of strategic deception in LLMs using the Werewolf game. We run 6 different setups for a cumulative sum of 500 games with GPT and Claude agents. We demonstrate that state-of-the-art models perform no better than the random baseline. Our findings also show no significant improvement in winning rate with two werewolves instead of one. This demonstrates that the SOTA models are still incapable of collaborative deception.
Cite this work:
@misc {
title={
Werewolf Benchmark
},
author={
Luhan Mikaelson, Zach Nguyen, Andy Liu, Jord Nguyen, Akash Kundu
},
date={
7/1/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
