Agentic Surveillance Mx

- The problem being tackled is very interesting: biases that cannot be detected through an output filter. The conceptual grounding in regulation is also a strong contribution.

- It's a real strength that the design includes different combinations of conditions, making it reusable for other contexts.

- The paper presents its central question as: "What happens when an agent doesn't mention any protected data in its output and still discriminates systematically?" However, based on the methodology, the question being actually tested is more specific: what happens when a system (model) is caught between two instructions — following a law versus following an existing procedure that already contains bias. These are two different problems and approaches. For the first question (spontaneous, undetectable bias), one could design an evaluation without giving the model legal context or the scoring rules upfront.

- The abstract is not clear. An abstract should present, in one paragraph, the problem, what was done, and the main results, but in this case it is difficult to understand what was actually done.

- The document is confusing to read: it has issues with writing/wording, spelling, and the ordering of content doesn't help the reader follow along.

El proyecto aborda un riesgo muy relevante para la seguridad de la IA: que un agente basado en LLM pueda producir una decisión discriminatoria sin mencionar explícitamente datos personales protegidos. Ese punto es especialmente valioso porque muestra una limitación de los controles centrados únicamente en PII visible: una salida puede parecer formalmente limpia y, aun así, incorporar una inferencia discriminatoria mediante proxies o razonamientos indirectos.

Una fortaleza del trabajo es que no se queda en una afirmación abstracta. El caso de reclutamiento permite mostrar de forma concreta cómo puede operar la discriminación por proxy en decisiones de alto riesgo, y el diseño escalonado ayuda a observar el efecto del idioma, el marco jurídico y el contexto mexicano. También es positivo que el equipo reconozca el carácter exploratorio de los resultados y las limitaciones del tamaño muestral, lo que le da seriedad metodológica al proyecto.

Como siguiente paso, el proyecto podría profundizar no solo en si aparece discriminación por proxy, sino en qué parte del diseño del agente la activa o la amplifica. Para una auditoría institucional sería muy valioso saber dónde intervenir: si en la selección del modelo, la función objetivo, el system prompt, los criterios usados como proxy o la forma en que el agente pondera esos elementos. Incorporar pruebas adicionales que varíen algunos de estos componentes permitiría pasar de detectar el riesgo a orientar mejor su mitigación.

Desde la perspectiva de gobernanza y rendición de cuentas, el hallazgo central es muy importante: la ausencia de datos personales visibles no equivale a ausencia de discriminación. Si agentes LLM se usan en decisiones de alto impacto como reclutamiento, salud, crédito o acceso a servicios, la auditoría no debería limitarse al resultado final. Debe poder revisar la cadena completa de decisión: modelo elegido, objetivo asignado, restricciones del prompt, criterios de clasificación, trazabilidad del razonamiento y rol de la supervisión humana. Ese puente entre evaluación técnica y responsabilidad institucional podría convertir el proyecto en una herramienta con mayor potencial para auditoría de IA en contextos latinoamericanos.

Clear engagement with a real measurement gap.

You asked a sharp question: can an agent violate privacy without ever mentioning a protected attribute in its output? The worked example in Box 1, which walks through resume_02 and resume_10 side by side with exact point allocations, is the strongest part of the submission. It makes indirect inference visible in a way the abstract alone can't.

One area to push further is the gap between what the staircase design was built to test and what it actually shows. None of the three comparisons in Table 6 (language, legal framework, culture) reach significance, so the variable you set out to isolate doesn't produce a measurable effect. The result that does hold up, model choice as the dominant factor, ends up reading as a secondary finding in section 4.4, when it's really the paper's main result. Reframing around that would make the contribution land more clearly.

The number of episodes attributed to claude-opus-4.8 in section 4.4 doesn't match your own repository. The paper reports 96 episodes with 94/96 judge confidence for that model, but your README states the full experiment is 96 runs total, covering 3 models times 16 scenarios times 2 replicas. One model alone should be 32 episodes, not 96. This is worth fixing since it's the number behind the headline claim.

The abstract and conclusion present the 0% vs. 100% gap as settled, but the limitations section says plainly that no Fisher test reached significance with n=2 per cell. A reader who only sees the abstract walks away more confident than the data supports.

One question: in Table 3, gpt-5.5 is noted as having exposed names that reveal gender in one replica. Was that scored as its own violation type, or folded into the indirect_inference count?

I wasn't able to run the Docker pipeline myself, since I had no API key and no local execution, so I can't independently confirm the violation rates beyond the CSVs you cite. I read both repositories, including the judge and analysis scripts, and the implementation matches what's described in the paper.

Overall, a well-instrumented experiment with a genuinely useful worked example, but the abstract claims more certainty than the body delivers, and the episode count needs a fix.

The text presents a novel and ingenious strategy but has a significant area for methodological improvement, in many cases highlighted by the authors themselves. They could also utilise open and local models to address potential biases in GPT-4.1. Conceptually, it appears that the authors confuse ‘discrimination’ with ‘privacy issues’. Whilst it is clear that these may be linked, the text sometimes refers to one and at other times to the other, as if addressing both issues simultaneously in an unclear manner without establishing the conceptual connection between them. As for the stylistic presentation of the work, it could be substantially improved; it reads like a first draft. A very good idea is overshadowed by carelessness. To improve the text, it is necessary to clarify what the scenarios consist of. The text contains sections in English and others in Spanish, which may be interpreted as a sign of carelessness. I think it is important to point out that the authors observe that model evaluation is predominantly carried out in English; however, they use English words and acronyms when Spanish equivalents exist.

The central finding is one of the hackathon's most relevant contributions for Latin American contexts, as it exposes exactly the type of violation that current PII filters cannot detect. The staircase design to isolate language, legal framework, and cultural context is methodologically beautiful :) . To consolidate the work, the most urgent next steps are increasing replicas per cell (minimum 10 for robust inference), including open-source models widely used in LATAM such as Llama or Mistral, and validating the GDPR - LFPDPPP equivalence with Mexican legal specialists. It is also worth correcting the duplication of Tables 3 and 5, which are identical.

El proyecto muestra una clara evidencia sobre el objetvo planteado, sería bueno seguir intentando encontrar las implicaciones más allá de los marcos normativos del uso de estos algoritmos, es más complicado, pero podría traer soluciones más ajustadas al concepto de fairness