SafeSurveil-AIxBio: Runtime-Gated Genomic AMR Triage with Evidence Graphs and Grounded AI Sidecars

A prototype for AMR triage (E. coli + tetracycline) where AI-generated explanations are treated as "sidecars" or non-authoritative outputs that must pass fact-checks against persisted evidence before reaching the analyst. Includes an execution gate (allow/review/block), evidence graph, deterministic reasoning trace, and provenance tracking.

Why it matters: The design principle is sound. In any AI-assisted biosecurity or clinical workflow, the AI's explanation should be subordinate to the actual evidence, not the other way around. The AMR literature consistently warns about phenotype-genotype discordance and database-dependent interpretation. Building the audit and trust layer first, before optimizing the predictions, is the right order of operations.

What's strong: The safety architecture is well-designed and the system was fully built and tested as software. Provenance tracking, evidence graphs, citation checks, and explicit fallback labeling are practical features. The dual-use appendix is unusually thoughtful for a hackathon.

What's missing: All the evidence that the system works is engineering verification (tests pass, builds succeed, the proof run completes). There's nothing showing it helps analysts make better decisions, catches signals they'd otherwise miss, or even performs comparably to existing tools. One organism-drug pair, fixture-trained baseline, no user testing. The contribution also isn't clearly distinguished from standard clinical decision support design. Treating ML outputs as non-authoritative pending human review is essentially how the FDA already approaches AI-assisted diagnostics.

Info hazard: Low–Moderate. Defensive tooling. Main caution is avoiding making uncertain AMR predictions look more decisive than they are, which the system is specifically designed to prevent.

I appreciated this submission's emphasis on observability and traceability. I agree with the authors that these are important properties in a software system deployed in a public health context.

I found that the report was quite dense and a little too focused on implementation details and architecture (It's possible that I am missing things about the importance here!). I would have liked to see more of a direct focus on the value add for public health decision making. It's a little bit unclear to me how this is tailored for the specific problem of AMR triage: I can imagine that AI models tend to be overconfident for many (most) decisions. If there is a novel aspect here for the XAI component perhaps I'd have liked to see just that component demonstrated and validated etc.

I think the main problem was something like ‘AI predicts AMR too confidently’, and the solution was an AI tool that understands when it is being too confident, and can modulate its output?

The Abstract is filled with jargon and felt very LLM-y. It was mostly incomprehensible to me. An Abstract needs to be relatively straightforward, and clearly define the background, problem, methods, top line results and conclusion.

The introduction follows in a similar vein: ‘Antimicrobial resistance is a critical biosecurity problem because the signals that matter for routine stewardship (resistance genes, phenotype predictions, mobile elements, and surveillance provenance) dictate how quickly an analyst can identify a high-risk case’ ← I have literally no clue what this means. The ‘Main contributions’ are similarly opaque, and in general the whole report felt riddled with overly technical LLM-speak.

The ‘ML for AMR and stewardship’ section was a little clearer, with reference to systemic review that I understood. I was glad to see that you mention a limitation of your method (or rather what it is not), but I was unable to understand what your tool’s advantage is.

The Methods were highly jargon-laiden (‘local-first E. coli…evidence factory’...’fixture-trained smoke baseline’....the list goes on). And the Results were similarly incomprehensible to me.

I would really recommend trying to write this report from scratch, without using an LLM. The text is riddled with technical jargon and the sentence constructions are very clearly LLM generated. The fundamental idea of using AI tools to assess AMR threat is obviously a good one, but I didn’t get a sense of how important your specific problem is and I had really no idea what was outlined in this report.