Nov 25, 2024
Investigate arithmetic features in Multi-lingual LLMs
Akash Kundu, Ashish Rai, Suhas K R
We investigate the arithmetic related feature activations in Llama3.1 70b model across its 8 supported languages. We use arithmetic-activation strength to compare the 8 languages and unsurprisingly English has the highest strength and Hindi, Thai score the least.
Jason Schreiber
Great research question. I find the intersection of math problems with its relatively clear evaluation criteria and multi-linguality a cool test bed for evaluating the robustness of feature steering. I hope the authors will iterate on this since it seems a worthy avenue!
Mateusz Dziemian
Good initial work! Definitely interesting to see that different languages have lower activations in such a universal topic like maths. Would be interested to see the difference in language dependent and in-dependent features on other languages and math benchmarks.
Alana Xiang
This team finds some features that activate on GSM8K. They made an interesting decision to compare across languages.
With more time, I'd love to see this team investigate why they were unable to improve the performance of the model via steering.
Cite this work
@misc {
title={
@misc {
},
author={
Akash Kundu, Ashish Rai, Suhas K R
},
date={
11/25/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}