Quite surprising results! Wasn’t expecting the model to have features which it doesn’t activate enough during inference that could boost performance. The results are very intriguing and warrant a further study to get a better understanding on the trade-offs for such features and if they just improve maths performance or other domains too.

It seems strange to use Google Translate or Mistral for translation when there are much better options available. This is especially problematic for challenging math problems. There’s also an inconsistency in Figure 1, where the numbers add up to 101 despite stating that there are 100 problems. French translations worked much worse, and I’m afraid the translations might not have been accurate. The text didn’t clarify it fully, but it seems they used Mistral for French and Google Translate for Russian. In general, the finding is interesting: they ran a math benchmark, used a contrastive method to find differences between correct and incorrect math answers, and checked if these differences transferred between languages—and they do. This provides evidence that the vector truly captures some sense of mathematical accuracy. That being said, they only evaluated the steering vector on samples where it was incorrect without steering. While this fixes some outcomes, it’s possible this also breaks previously correct responses. In French, it only corrected 2 out of 21, and I’d guess that some false answers might arise just from resampling. They also applied it to only 21 out of 77 false samples. I would also find it more interesting to take the correctness vector for a language in which the model performed better. Using a control feature and validating on all math problems could be beneficial too. You’d expect better results if you resampled on false outputs, so it would be interesting to see what happens if you steer for an unrelated feature.

This is creative paper which finds a new domain on which SAE features generalize well (across languages in grade school math).

The surprising finding that the French steering vectors had a larger impact on English and Russian performance than French performance warrants further inquiry. I think this paper could've significantly improved on novelty if it pursued this direction.

Given more time, I would also love to see the authors inspect whether the features they found generalize beyond GSM8K.

Good work!