Guardian LATAM: Early Detection of Hallucination Risk in Spanish-Speaking Multi-Agent AI Systems Using Consensus Geometry

Detecting hallucination is important. The value add of the new dataset isn't clear to me, as there are many hallucination-oriented benchmarks already. It's not clear whether the classification algorithm works, because you only reported performance on the new dataset, and didn't present the performance of a baseline.

The Spanish-language benchmark you built is the strongest part of this submission — 120 prompts grounded in Colombian civic, legal, and educational terminology (Registraduría, CONPES, acción de tutela, debido proceso) is a real artifact other Spanish-language safety researchers could reuse. The four-tier × six-domain structure is a clean design, and the honest, specific Limitations section plus open-source reproducible scripts raise the floor of the work.

Guardian LATAM is a lightweight hallucination-risk monitoring layer for Spanish-speaking multi-agent AI systems that estimates risk from five consensus signals (semantic divergence, majority cohesion, confidence spread, contradiction indicators, escalation phrases) derived from independent agent responses, avoiding the cost of a separate LLM judge. The authors introduce a purpose-built 120-prompt Spanish-language benchmark organized into four risk tiers across six civic and legal domains, reporting 90% overall accuracy and 100% on hallucination-trap and safety-escalation tiers. The motivating context — resource-constrained Latin American institutions that cannot afford expensive judge pipelines — is a genuinely underserved deployment scenario in the AI safety literature.

Strengths

1. The LATAM problem framing addresses a real and underserved gap. Safety evaluation infrastructure is overwhelmingly English-language and resource-intensive; targeting Spanish-language multi-agent deployments in civic, legal, and healthcare contexts gives this project immediate practical stakes.

2. The four-tier benchmark design (Known Facts → Uncertainty → Hallucination Traps → Safety Escalation) maps cleanly to a realistic threat model, and the REFUSE_OR_ESCALATE routing output goes meaningfully beyond binary hallucination detection toward an actionable safety interface.

Weaknesses

1. The risk-score weights in Eq. (3) are tuned on the same 120 prompts used to report 90% accuracy — there is no held-out test split, so the headline figure is an in-sample fit, not a generalization estimate. A simple fix: reserve 25–30 prompts before any tuning and report accuracy on that held-out set.

2. The perfect 100% scores on Tiers C and D are circular by design. The Tier D prompts were built with Spanish escalation phrases that trigger the system's hard-coded escalation override, and the Tier C prompts were built to produce agent disagreement — so these results confirm the triggers fire correctly, not that the system generalizes to hallucinations it wasn't designed around.

3. No baselines are included, so the 90% overall accuracy cannot be interpreted as evidence for Consensus Geometry over simpler alternatives. A keyword-only classifier or majority-vote baseline on the same agent outputs would immediately contextualize the figure.

4. The system's validity depends on the three agents producing independent responses, but their underlying models are never identified. If all agents share the same LLM, responses will be correlated and the divergence signal may be systematically suppressed regardless of actual hallucination — a direct threat to the core mechanism that can be addressed simply by disclosing model names and versions.

Guardian LATAM addresses a relevant need in low-resource contexts. Specifically, detecting hallucination risk in Spanish multi-agent systems without relying on an expensive LLM-as-a-judge. The consensus geometry framework is clearly explained and the lightweight implementation makes practical sense for public and educational institutions in LATAM. However, the work's most important limitation is not explicitly discussed. I see an issue in this: the benchmark was constructed by the same author who designed the system, with heuristic weights tuned on that same benchmark (which makes the 90% accuracy difficult to interpret as evidence of generalization). To give the results credibility, the most urgent next step is evaluating the system on an independent external benchmark, or on a held-out subset excluded during design. Additionally, replacing token-overlap similarity with multilingual embeddings (Sentence Transformers) and activating the Confidence Spread feature with calibrated agents would make the system substantially more robust.