Investigating Activation Threshold Failures in Cross-Lingual Prompt Rejection

This is one of the more technically grounded submissions in the hackathon. The core contribution — breaking down cross-lingual safety degradation by semantic domain rather than treating it as a uniform effect — adds genuine value beyond prior work. The finding that Economic Harm suffers severe Silhouette degradation in Portuguese while Illegal Activity remains almost structurally unchanged is immediately actionable for practitioners prioritizing where to patch multilingual safety. The KDE plot communicates this well.

Three concrete suggestions for strengthening this work:

Report sample sizes per category. The per-domain Silhouette Scores and magnitude results are the paper's central empirical claims, but the number of prompts per category is never stated. With small per-category samples these scores can be quite noisy, and this is the first thing a reviewer will ask. Adding this to any follow-up write-up is essential.

Validate the Layer 15 choice empirically. The theoretical justification for Layer 15 is well-argued, but a brief comparison across a few layers (e.g. 10, 15, 20, 25) would show whether the magnitude collapse is strongest there or whether a different layer tells a cleaner story. This is a cheap experiment that substantially strengthens the methodology.

Implement the inference-time steering intervention. The introduction proposes zero-shot latent steering to restore Portuguese magnitude as a potential fix — this is actually the most original idea in the paper — but it was cut to future work. Even a proof-of-concept on one domain (e.g. boosting Economic Harm prompts in Portuguese to match English magnitude) would make this a complete contribution rather than a diagnostic study. That experiment alone could be a strong short paper.

Overall: the mechanistic framing is right, the domain-specific breakdown is a real contribution, and the use of A100 compute with TransformerLens for proper activation extraction shows serious technical execution for a hackathon context. With the steering experiment implemented and sample sizes reported, this is well worth developing further.

This is a technically strong and relevant project that empirically probes cross‑lingual safety in Llama‑3 via mechanistic analysis of residual stream activations, introducing “Magnitude Collapse” and domain‑specific “Boundary Collapse” between English and Portuguese refusal behavior—highly pertinent for Global South contexts where low‑resource languages are common. The methodology is well crafted for a hackathon: curated dual‑use prompts across harm domains from established multilingual safety datasets, deliberate choice of Layer 15 as a semantic midpoint, explicit construction of a universal refusal direction, and use of scalar projections, KDEs, and silhouette scores to quantify geometric decay.

The authors clearly acknowledge limitations (single model family, one language pair, exploratory scope), but the work would be stronger with clearer practical mitigation pathways (e.g., a brief concrete example of latent steering at inference time), a bit more intuition for readers unfamiliar with mechanistic interpretability (short explanation of residual streams/refusal directions), and a sharper distinction between empirical findings and hypotheses.

Overall, it is a well‑executed exploratory study with high potential impact that convincingly shows safety alignment degrading in specific semantic domains across languages and could be further improved by tightening exposition for non‑specialists and sketching concrete next‑step interventions.

The core lens is a good one: decomposing the "universal refusal direction" into direction versus magnitude.

A few changes would substantially strengthen it. First, the headline "−371.6% magnitude collapse" is computed on a projection axis whose zero is arbitrary; the underlying effect is a −0.32 shift in mean projection, so reporting an absolute effect size would be far more credible.

Second, the safety claim is entirely geometric and, by your own framing, "theoretical" - a behavioral test showing the magnitude collapse actually produces a jailbreak would close the loop.

Third, the framing is low-resource fragility, but Portuguese is a high-resource language, which may itself explain the directional robustness you found; a genuinely low-resource language would better match the premise.