TUP Detection: Hybrid Prompt-Injection Guard for AI Generative Security Monitoring

A clean, well-executed engineering submission — code, paper, demo, reproducible caches, all present and coherent. The work is honest and disciplined: clear research questions, a proper ablation, frozen score caches for offline reproduction, and a limitations section that flags every weakness a reviewer would raise. But there's a tension the paper itself surfaces and doesn't fully resolve, and it's central to how much credit the headline deserves.

The hybrid's contribution over Sentinel-alone is marginal — and the paper is honest that it is. Table 1 is the tell: Sentinel-only and Hybrid both land at 95.1% PINT. The entire value-add of Layer 1 is +3 true positives (245→248) bought at +4 false positives (12→16). So the "95.1%" headline is essentially Sentinel v2's number, not TUP's. The genuine contribution is narrower and should be framed as such: explainable rule_id attribution on a handful of catches, not a accuracy gain. The discussion concedes this ("upgrading the classifier backbone... accounts for the largest measured gain"), which is admirably candid but also means the system's own novel layer is doing very little.

The headline number is a wrapper around someone else's model. Sentinel v2 is pre-trained, not fine-tuned ("no fine-tuning" is stated as a feature). That's a legitimate engineering choice, but it means the empirical result is "a good off-the-shelf classifier scores 95% on deepset," with TUP providing orchestration. The +22.8pp over DeBERTa is really "Sentinel v2 > DeBERTa," not "TUP's architecture > baseline."

Baseline comparisons are explicitly not head-to-head. The Sentinel-card ~88% and ProtectAI 77.6% figures come from different splits/thresholds and weren't re-run. The paper says so plainly and puts "re-run baselines under our infra" on the roadmap — correct posture — but it means Table 2/Figure 2 shouldn't be read as a real comparison, and the abstract's "+7pp over Sentinel v2" is the weakest claim in the paper.

Crescendo 100% is real but small and favorably configured. n=10, and the 100% comes from full-transcript scoring — which feeds the whole conversation in. The more operationally realistic stateless setting gets 75%. The paper flags both, which is right, but the headline "100% multi-turn recall" rests on the easier setting and a tiny sample.

τ=0.15 inflates recall at FP cost. Benchmark mode is explicitly tuned for max recall and isn't the production threshold (0.5), so the headline numbers aren't the deployed behavior. Disclosed, but worth keeping front-of-mind.

The project tackles an important problem with a reasonable architectural approach, but it is currently at a proof-of-concept stage rather than a validated production system. I would agree with the project if the authors strengthen it with broader dataset evaluation, statistical testing, and production-mode benchmarks.

The project comprises an OWASP plus Sentinel-based prompt-injection identification engine. The team has saved their results, making it reproducible, which is the biggest win. However, the team states that the AI classifier used already has a ~95% accuracy, so the rule-based system doesn't add a significant boost. The ~22% boost specified is due to using a better model, so it doesn't clearly explain how the rule-based classifier improves the model.

Overall, it is a great start toward building an identification engine, but it needs more detail and real-world benchmarks to make it a production-ready system.