Nov 2, 2025
Does the direct method predict general capability
Emil Schmitz
Epoch AI's direct method assumes that lower average loss indicates better general capabilities. We posit that the loss may possibly be indicative only of higher performance on specific content. We attempt to prove this by calculating loss on high-level chess games. To calculate loss, we compare the LLM's prediction to those of open-source chess engine Leela-Zero.
At the time of submission, the experiments have not yet run through. I will try to finish them and notify you, if that works.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Does the direct method predict general capability
},
author={
Emil Schmitz
},
date={
11/2/25
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


