Do Multilingual Vision-Language Models Abstain under Cross-Modal Conflict in Low-Resource Languages?

This is a technically strong and highly relevant AI safety project. The paper addresses an important gap in multilingual VLM safety by studying how models behave under cross-modal conflict in low-resource languages. I particularly liked the perception-control design and “override share” metric, which separates genuine caption-driven failures from baseline visual perception issues. The work is also strengthened by evaluation across nine VLMs, three languages, and four datasets, plus a mechanistic probing and cross-lingual steering component.

To improve the work further, I would recommend expanding beyond Hindi and Telugu, adding more frontier or closed-source models if possible, and giving deeper treatment to the perception cost introduced by steering.

Great work exploring the breadth of application of LLMs for languages that typically get less attention and compute on 2B to 9B models. Would be great to also see if this interpolates to larger models as a next step

While execution was strong, it could have used some more techniques to be more comprehensive, considering potential limitations of tokenization

while presentation was logic, with graphs to illustrate, would be great to see more step by step details

Great submission and very well thought out. One thing that could make it even stronger is explaining more clearly how the multilingual examples were checked, since translation and cultural context can affect the results. It would also help to explain the tradeoff between making models more cautious and making them too cautious.

The core insight that English only safety evaluations of VLMs create a blind spot for hundreds of millions of low-resource language speakers is important and under explored. The causal story around language resource level would be significantly strengthened by controlling for concrete variables like per-language training token counts and tokenizer fertility, rather than relying on a coarse literature-based ordering of English > Hindi > Telugu. Presentation-wise, the paper would benefit from a cleaner motivating example for the override share concept early on

This is a strong and well-scoped technical AI safety submission. The project identifies an important multilingual deployment risk: VLMs may behave safely under image-text conflict in English but degrade in lower-resource languages through either text capture or excessive abstention. The perception-control design and override-share metric are especially valuable because they separate genuine caption-driven override from baseline perceptual failure, leading to a more informative model safety ranking. The accompanying codebase strengthens the submission: it includes an end-to-end benchmark pipeline for dataset construction, multilingual VLM inference, perception-control scoring, result aggregation, hidden-state probing, and activation-steering experiments. The evaluation across nine open VLMs, three languages, and several domains is impressive for a hackathon. In particular, the COCO-derived natural-scene set gives the benchmark a useful grounding in realistic image-caption conflicts, though its construction from counterfactual/minimal-edit caption pairs and heuristic MCQ generation means that further human validation would be important before treating it as a mature benchmark.

That said, the broader area of cross-modal conflict, VLM hallucination, multilingual model robustness, and inference-time steering is already well studied in the research literature, so the project’s novelty is best understood as a strong combination and extension of existing threads rather than a wholly new research direction. The probing/steering component adds a promising mechanistic angle, but the main limitations are that the MCQ abstention setup may not fully reflect open-ended deployment behavior, some per-domain/language cells appear small, the COCO-derived examples may inherit biases or artifacts from the source dataset and generation pipeline, and the steering mitigation has a real perception/over-abstention tradeoff. Overall, this is a high-quality, clearly safety-relevant project that others could build on with larger datasets, free-form generation tests, more languages, stronger dataset validation, and reproducibility checks. I encourage the authors to continue developing this work, as it has a clear path toward becoming a useful multilingual VLM safety benchmark and audit methodology.