Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts

Our results demonstrate that public benchmark scores do not always accurately assess model properties, and underscore the importance of improved data practices in the field.

Paper

Code

Explainer