The Pix/CPF False-Positive Problem in LLM Fraud Detection

The paper ships a contrastive benchmark of 914 Brazilian-Portuguese message pairs. The categories cover Pix scams, CPF and tax document fraud, fake bank central impersonation, QR code tampering, WhatsApp cloning, fake benefit notices, and adjacent social engineering patterns. Each harmful message gets a benign counterpart that holds scenario, vocabulary, and length constant, so the pair differs in intent and not in surface lexical cues. The harness asks an LLM fraud analyst (liquid/lfm-2-24b-a2b) to return a structured verdict. Pair-level accuracy isolates whether the model reasons about intent or just reacts to fraud-coded topic words. What jumped out is the asymmetric failure mode: 95.9 percent harmful-side recall against 19.5 percent benign-side specificity, full pair accuracy of 15.3 percent. Benign no-manipulation messages (53.2 percent) score worse than every actual manipulation tactic except social proof.

The framing and the category taxonomy carry the work. Pix and CPF as a deployment context is well motivated and underrepresented in published safety benchmarks. The category list (pix_bug_scam, qr_code_tampering, fake_bank_central, income_tax_refund_scam, fake_benefit_scam) mirrors fraud patterns that BCB and Febraban publish in operational risk reports. The contrastive-pair design is the right call for this question, because standalone classification cannot separate topic overfitting from intent reasoning. Releasing the full JSONL dataset, the contrastive pairs, the evaluation harness, the analyst prompt schema, and an evaluation report makes the work reproducible end to end. That is rare for a hackathon submission. Flipping the safety frame from "model misses fraud" to "model over-flags legitimate communication" names the real product risk in Brazil. Alert fatigue erodes user trust in the channel itself, and a Pix filter that cries wolf on every IPVA notification trains users to swipe past the actual scam.

Three additions would strengthen the work. The first is the most load-bearing. The same model family generates the harmful messages, writes the benign counterparts, and then evaluates both sides. The limitations section acknowledges this but does not quantify it. A partial replication using human-written benign messages (redacted bank notification corpora, BACEN consumer protection samples) would separate the topic-overfitting signal from a generation-evaluation coupling artifact. One thing worth flagging: several benign counterparts in the released pairs still ask the user to confirm a password or click a verification link under urgency. Brazilian banking standards under BCB and Febraban guidance treat that as phishing, so labeling these as benign inflates the false-positive rate the paper reports. Pair id bd492e68 is a clean example, where the benign side still says "verifique sua senha no app até as 18h". A real BRB or Itaú fraud team would tag that as scam. The second addition is methodological hygiene on the sweep. The five-model pilot runs on 10 to 15 pairs per model, and the overall.accuracy field in each sweep report is copied from the primary-model evaluation, not recomputed per model. The root cause sits in scripts/run_model_sweep.sh, which calls evaluation.analysis without --results-path, so analysis.py falls back to DEFAULT_RESULTS_PATH for the "overall" block while only "contrastive" recomputes per model. Worth pushing back on the cross-model "pattern holds" claim until that is fixed. Present the sweep as exploratory, omit or recompute the per-model overall numbers, and run the full sweep over the 914 pairs against models that did not generate the dataset. The third addition is institutional. A partnership with a Brazilian regulated entity (a bank security team, a BACEN or Febraban working group, the SERASA fraud panel) would supply a real distribution of legitimate Pix and CPF notifications, which is currently the weakest leg of the benchmark.

The broader signal is the most interesting contribution. Most LLM safety literature frames the risk as missed harmful content. This work documents the opposite failure mode in a regulated financial setting, where over-blocking is itself a safety and trust harm. That framing connects naturally to misinformation work on over-moderation and to deployment audits for content filters operating on culturally embedded vocabulary. The methodology extends to other Global South payment rails (UPI, M-Pesa, SPEI), where the same topic-overfitting pattern is plausible and currently untested. For a weekend hackathon, the alignment of an underrepresented domain, a methodologically defensible contrastive design, an honest limitations section, and a fully released dataset and harness makes this a project worth following up on with a regulator-validated dataset and a full multi-model sweep.

This one matters well beyond the benchmark, because Pix isn't staying in Brazil. India's UPI is already there, Colombia's Bre-B is rolling out, and every one of these instant-payment rails inherits the same problem: the scams wear the same government-and-bank language as the real notices, and the fraud filters bolted on top get deployed into markets the models were never tuned for. So a fraud analyst that over-flags legitimate Pix and CPF messages isn't a Brazil-only bug, it's a preview of what breaks everywhere this model ships. That's what makes your result worth taking seriously.

That's the version nobody writes: you expected the model to miss disguised fraud, found the opposite, and reported that instead of bending the data back. It catches real scams at 95.9% but clears legitimate Pix/CPF messages only 19.5% of the time, and benign messages with no manipulation tactic score worse than almost every fraud technique (53.2%). That's the tell it's reacting to words like "Pix" and "CPF," not intent. The twin example in the appendix, one CPF message threatens suspension, the other confirms all is fine, lands the whole thesis in one read.

Where it's thin is one flaw doing double damage: the same model writes the benchmark and grades it. You flag it twice, so you feel it, and it's load-bearing. A model over-flagging benign text it wrote itself can't separate "models over-trigger on fraud words" from "this model can't read its own benign half."

Two fixes: split generation from evaluation so a different family judges than writes, and run the full 914 pairs across your model set instead of 10 to 15, since that pilot carries the "every tier does this" claim and can't hold it yet. Significance testing on that sweep turns your strongest sentence from suggestive to proven.

One cleanup: §4.3 shows a category table, then says nothing can be drawn from it. Cut it or justify it. The finding is worth proving, and it travels well past Brazil.

For a nation that relies heavily on digital payment rails like Pix for daily life, an AI filter that is this trigger-happy poses a significant systemic risk. It directly leads to alert fatigue and erodes public trust in legitimate, routine communications from banks and government entities. We cannot accept safety evaluations that systematically overstate how safe a system is for users in markets where scams flawlessly mimic ordinary financial correspondence. This research is an essential step forward. It proves that to establish meaningful algorithmic liability, we need benchmarks tailored to the cultural and infrastructural realities of the Global South. Expanding this contrastive methodology to other regional systems, such as M-Pesa in Kenya or UPI in India, will be vital for holding these models accountable on a global scale.