SwahiliGuard An AI-Powered Safety System for Detecting Localized Online Harm

I like the apporach, I like how there is sepeartion in text and audio, done for a hackathon. Sovereignty in AI is key and this approach has helped companies like Sakana and Sarvam, I encourge you to use this work and actually go build a company around it.

You understand the limitations, you tried too much during a hackathon but focus on voice and go deep and then go after text, voice data is a bottleneck today but if you are able to source that you have a great asset with you.

One-line summary: A Swahili-aware content-safety classifier for cyberbullying, hate speech, and gender-based violence, built by fine-tuning XLM-RoBERTa to output three risk tiers (Low/Medium/High), with claimed explainability and voice-reporting components, reporting a Macro F1 of 0.86 and under 4% false negatives on high-risk content.

Constructive critique:

The problem is real and the team clearly knows the domain. Swahili is genuinely under-served, the focus on GBV and code-switched harassment matters, and several design choices are well-reasoned: choosing XLM-R for transfer learning on limited data, preserving character-level obfuscations instead of aggressive stop-word removal, class-weighting for imbalance, and a three-tier rather than binary output so moderators can triage. The "What Did Not Work" section is the best thing in the paper. Reporting that TF-IDF plus SVM collapsed on Swahili verb morphology, and that generic stop-word lists destroyed the signal, is exactly the kind of honest, specific lesson that shows real experimentation happened.

The problem is that almost nothing in the Results survives scrutiny, because there are barely any results to scrutinise. The entire empirical case is a single number, Macro F1 0.86, plus "under 4% false negatives," with no dataset size, no train/test split, no confusion matrix, no per-class precision and recall, and no baseline figure. That last omission is the most damaging: the central claim of the whole paper, that specialised fine-tuned models catch what generic multilingual models miss, is stated as an observed trend but no comparison is ever reported. You assert the headline finding without running the experiment that would prove it. On top of that, the evaluation appears to run on "simulated" data the team generated themselves, and an F1 measured on synthetic text drawn from your own distribution tells you very little about real Swahili harm in the wild. Hamel Husain and Shreya Shankar's point lands hard here: a clean aggregate metric with no error analysis on real traces is the classic way to fool yourself, and for a safety classifier the risk tiers you defined are the eval, so they need to be validated against human labels, not against your own simulator. Two of the three headline contributions, the explainability layer and the voice-reporting pipeline, are described only in "designed to" and "supports" language, never shown with a single example, and the limitations section effectively concedes the voice component was not built or tested. So the paper claims three contributions and substantiates roughly half of one.

Three fixes. First, report a real results table: dataset size, split, per-class precision and recall, a confusion matrix, and crucially a generic-XLM-R baseline next to your fine-tuned model, because without that the core claim is unsupported. Second, evaluate on real annotated Swahili, not self-simulated text. A relevant resource already exists and you even cite adjacent work (PolitiKweli is a real Swahili-English code-switched dataset), so the absence of any real-data evaluation is a notable gap rather than an unavoidable one. Third, cut the rhetoric and the repetition hard. The Results, Discussion, Future Work, and Conclusion sections restate nearly the same paragraph four times, the Future Work section contains no future work, and sweeping lines like "challenge the industry standard" and "true alignment cannot be achieved globally unless" are doing work the evidence has not earned. Also clean up the formatting: the LaTeX math markup is rendering as literal dollar signs throughout.

Track + flags:

On-topic (Global South AI safety, Africa, harm detection). LLM use disclosed (Gemini), references are real and relevant, no plagiarism signals. Note for the panel: the headline metrics (F1 0.86, <4% FN) are reported with no supporting methodology and appear to be measured on self-generated simulated data, so they are effectively unverifiable, and two of the three claimed contributions (explainability, voice) are not demonstrated.

Strengths: Focuses on an important underserved language ecosystem and addresses a meaningful AI safety challenge. The problem statement is compelling and socially relevant. Areas for Improvement: The technical methodology requires additional detail, particularly regarding model development, evaluation datasets, and validation methodology. Expanding testing across diverse linguistic scenarios and presenting quantitative performance results would strengthen confidence in the solution's effectiveness.