HERRAMIENTA DE EVALUACIÓN Y RECOMENDACIÓN PARA LA PROMOCIÓN DE USO RESPONSABLE DE IA EN PYMES LATINOAMERICANAS

This is one of the most thoughtfully designed projects in the set. The core architectural decision - route everything with a normative consequence (gap severity, maturity scoring, control mapping) through a fixed deterministic rules engine, and let the LLM enter only at the end to draft the policy - is exactly right, and it directly defeats the "cosmetic compliance" failure mode where two firms with identical answers get different diagnoses on each run. The three-layer anti-hallucination defense (abstention below a 0.35 relevance threshold, automated citation-existence verification, and a second-pass claim-support check) and the hybrid RAG with reciprocal-rank fusion are genuinely well-engineered for a hackathon. Mapping all 30 scorable nodes simultaneously to NIST AI RMF, ISO/IEC 42001, and Floridi's five consolidated principles is solid, and the framing of the problem (99.5% of LATAM productive units are MSMEs for which NIST/ISO are impractical, existing regional efforts are static checklists) is well-evidenced.

The main limitation is validation, which the team handles with unusual intellectual honesty. (1) There is no field validation - a single constructed theoretical case can demonstrate intended behavior but not external validity. The highest-value next step is a small pilot (even 5-10 real MSMEs) reporting whether the 30/90-day action plans were actually actionable. (2) The 90 node-to-axis weights were LLM-generated and only a sample was hand-audited, yet these weights drive the entire diagnosis; an undetected wrong mapping silently distorts every score. Audit the full set, or at minimum report the audited fraction and the error rate you found in the sample (you note you already caught incorrect control citations - that finding deserves quantification). (3) No inter-rater concordance on the node-to-control/principle mapping; agreement between two domain experts on a subset would substantiate the coverage claim. (4) The RAGAS metrics (recall@6=0.85, citation precision=1.0, faithfulness=0.884) are encouraging but rest on a 10-query golden set - expand it and report per-query so stability is visible.

Concrete fix: run a small real-MSME pilot and fully audit the frozen weights. The design is strong enough to deserve, and survive, real validation.

The project addresses concerns especially salient for Latin America of how small and informal companies can comply with AI risk management frameworks. The focus on these specifically salient risks is smart. The paper is also admirably transparent about its limitations, clear distinguishing what the design demonstrates from what field validation would need to establish. The text sometimes introduces concepts with little contextualization, which undermines its clarity -- such as the reference to the "theory of the Drittwirkung of the Grundrechte" in the Technology, Ethics and Human Rights section. The proposed business model of the SaaS is not discussed, which is a meaningful gap for a tool targeting informal small companies. The walkthrough examples are also hard to follow, though this may partly reflect the machine translation from the original Spanish.