When Models Refuse to Speak: Task-Dependent Caste Bias in Smaller LLMs

This work examines an important and underexplored fairness issue by investigating how safety alignment affects the measurement of caste bias in LLMs. The central observation that explicit bias evaluations may underestimate underlying bias due to model refusals is valuable, particularly in the context of caste, an often overlooked axis of bias.

The paper would be strengthened by:

- More rigorously demonstrating that refusal behavior explains the explicit–implicit gap (e.g., by analyzing only answered prompts or conducting correlation analyses) in addition to inferring causality from descriptive statistics.

- Extending the evaluation to Indian languages, where caste associations are often more culturally grounded than in English.

This project audits caste bias in two open-weight Llama models (3.1-8B and 3.3-70B) using 48 IndiBias sentence pairs, comparing explicit (direct group comparison) and implicit (fill-in-the-blank) prompt formats. The central claim is that safety alignment silences caste bias rather than removing it: explicit prompts trigger refusals that hide the bias, while implicit prompts bypass those filters and surface stereotypical associations. The work is motivated by the deployment reality of Global South developers, who mostly ship accessible open-weight models rather than frontier ones.

Strengths

1. The deployment-motivated model choice is the project's strongest idea. Targeting accessible open-weight models — the ones most available to Global South developers — rather than frontier models gives the work a clear and practical safety stake that most benchmark studies lack.

2. The refusal-rate breakdown in Table 1 is a clean, legible result. Explicit prompts drew refusals (25% on 8B, 12.5% on 70B) while implicit prompts drew none, which makes the suppression mechanism easy to see without heavy statistics.

3. Pairing qualitative analysis with the numbers was a good instinct. The observation that the model can re-assign negative traits to marginalized groups is a promising lead that pure stereotype-score counting would miss, even though it is still an unquantified observation.

Weaknesses

1. At N = 48 per condition, the reported percentages are point estimates with no confidence intervals, and the differences between them are never significance-tested. This is clearest in the 2-point 70B explicit-vs-implicit gap, which is read as a real effect but at this sample size could easily be noise — reporting 95% binomial intervals and testing the key differences would show how much the numbers actually support.

2. The core framing — "bias is silenced, not removed" — is largely borrowed from prior work (Himelstein et al. 2026, Zhao et al. 2025), and a caste-specific implicit/explicit comparison already exists (DECASTE). The genuinely new piece is narrower — quantifying the refusal mechanism in small, deployable models — and framing the contribution that way would be both more accurate and more compelling.

3. The 0% implicit-refusal rate may be partly a prompt-format effect rather than the claimed task-recognition mechanism. The fill-in-the-blank prompt grammatically forces a caste-noun completion and leaves no room to refuse, so this alternative explanation should be named as a limitation.

This is an important AI safety issue, especially as LLMs see broader adoption in India and may be used by or about underprivileged communities where caste-related harms can have real-world consequences. The paper’s focus on caste bias is relevant and underexplored.

However, the core finding is not entirely novel: direct questions about protected or sensitive attributes are likely to trigger safety guardrails, while indirect probing can still surface latent stereotype patterns. The contribution is valuable in applying this framing to caste bias, but the evidence is not yet sufficient to support a broad claim.

The evaluation uses a small dataset, only a limited set of prompts, and only one model family. It also does not test the latest or a wider range of frontier and open-weight models. The claim that alignment hides rather than removes bias is plausible and important, but it needs more validation through larger datasets, more prompt variations, multilingual evaluation, base-vs-instruction model comparisons, and broader model coverage.