Multilingual Jailbreak Vulnerability Benchmark and Mitigation for Low-Resource African Languages
Godwin Abuh Faruna
We present a two-part study: (1) a multilingual jailbreak benchmark revealing large safety gaps in four open-weight LLMs across seven languages including Igala, which has no prior AI safety coverage, and (2) Latent Space Refusal Anchoring (LSR-Anchoring), a training-free activation-steering mitigation that recovers safety at inference time without retraining or target-language data. The combined result is a full pipeline: we find the failure, characterise it geometrically, and fix it, all without supervised African-language data. Across four models tested, English SRR stays at 85–99%, while African language SRR collapses to single digits in some cases. LSR-Anchoring recovers safety to near-ceiling on four languages at 70B scale, with MMLU accuracy drops below 0.35 percentage points. Arabic fails on all methods due to geometric misalignment, providing a clear deployment decision rule.
Very well executed. It goes beyond benchmarking to actually propose and implement a fix (training-free activation steering to recover cross-lingual safety). The work is ambitious and mostly delivers.
Strengths:
- Actually builds a mitigation, not just a benchmark. The LSR-Anchoring method works at inference time without retraining, which is practically useful.
- The Arabic negative result (English-derived steering makes things worse) is genuinely interesting and reveals something about the geometry of the refusal manifold.
Areas for improvements
- basically the limitations mentioned in the paper. Using a semantic classifier instead of phrase-based, multi-human review, using obfuscated prompts, etc
- slides missing
Great project. Can be expanded to solve multiple other problems within the space. Innovative and good execution quality but can be presented better.
The project evaluates different open source LLMs with different African languages searching for jailbreak vulnerabilities that are possible due to the model’s overfitting in English language information. The authors successfully the problem LLM’s have with these languages and offer a solution by using Latent Space Refusal anchoring to add the English refusal direction to the model’s response to counter the jailbreak while also considering that the jailbreak could be a consequence of model incapability to understand the language. However, the way that the mitigation approach works is not sufficiently explained in the document with missing terms in crucial equations. Likewise, the paper could use grounded examples of some prompts and responses with and without the jailbreak (while blurring sensitive data) to demonstrate how the proposed defense works practically.
Cite this work
@misc {
title={
(HckPrj) Multilingual Jailbreak Vulnerability Benchmark and Mitigation for Low-Resource African Languages
},
author={
Godwin Abuh Faruna
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


