REIA — Evaluador de Riesgo de Cumplimiento Legal en IA para Latinoamérica

This project addresses an important problem: organizations in Latin America are adopting AI systems faster than they can obtain reliable, affordable legal-compliance review. A tool that provides fast first-pass triage, cites legal sources, distinguishes jurisdictions, and makes compliance outputs reproducible could be useful for regulators, civil society, startups, and public-sector teams. The inclusion of quechua, aymara, guaraní, and plautdietsch is a valuable access-oriented feature and is the project’s most distinctive contribution.

The proposed architecture is sensible at a high level. Separating legal-source retrieval from legal analysis, restricting the model to cited sources, calculating the global score deterministically in code, and caching results by content hash are all good design choices. These choices respond to real risks in automated legal tools: hallucinated laws, unstable outputs, opaque scoring, and lack of reproducibility.

The main limitation is that the project is still mostly a design proposal rather than a demonstrated system. The Results section explicitly says that the final numbers must be confirmed by running the APIs and that the table represents the design of the demonstration.

A second limitation is legal reliability. The claim that the system cannot invent laws is too strong. Restricting the model to search-returned URLs reduces hallucination risk, but it does not eliminate legal error. The paper acknowledges some of this in the limitations, but the abstract and conclusion should be more cautious. A third limitation is that the submission appears unfinished.

The most important next step is validation. The authors could run the system on a set of test cases across multiple jurisdictions, compare its outputs with expert legal review.

Overall, this is a useful and socially valuable idea with a reasonable architecture, but the current version is preliminary.

The results are presented as designed verdicts, not measured ones, on three cases too obvious to prove anything. The central claim that the tool cannot hallucinate laws is overstated, since limiting it to cite only retrieved norms stops it from inventing a fake statute but not from misreading a real one or trusting a web page that is a blog rather than the official text. Its reproducibility is also weaker than claimed, because the cache just freezes the first answer, right or wrong, and changes if the project is reworded, so it gives consistency rather than reliability, and the final score still rests on uncalibrated numbers the model produces.

The legal foundation is the bigger problem. Most of the norms it judges against are still bills under review rather than enacted law, so its verdicts carry high uncertainty and can flag a project for breaking a law that does not yet exist. The hard-block rules are each tied to a single country, but the paper never explains how the selected jurisdiction determines which rule applies, leaving a core part of the logic unspecified, and the tool is never checked against a real legal answer, so it can be perfectly consistent and still wrong.

The report is well written, but the delivery is careless, with the tool named REIA in the title and LegisAI LatAm throughout the body, and the code and author links left blank.