Mar 22, 2026
Supply Chain Malware as Ground Truth: A Trace-Based Agent Safety Monitor Trained on Real Credential Exfiltration Patterns
Selim Sevim
Existing agent safety monitors evaluate tool calls in isolation, rely on keyword filtering, or require expensive frontier model inference per action. All three approaches share a common weakness: they lack labeled training data from real agent attacks.
We observe that this labeling problem has already been solved in an adjacent domain. Malicious PyPI packages execute credential exfiltration through the same behavioral sequence a misaligned agent would use: read environment variables or sensitive files, optionally encode the data, transmit it to an external endpoint. The behavioral trace is structurally analogous to what we want to detect in agents.
We repurpose DataDog's malicious-software-packages-dataset (2,339 packages, 2023-2024) as ground-truth training data for an agent safety monitor. A static AST parser extracts 339 verified exfiltration traces. We fine-tune Phi-3-mini (3.8B) with QLoRA on a balanced dataset, using an autoresearch-style hyperparameter search loop that evaluated 54 configurations overnight on a single GPU ($18 total compute).
On a 30-example hand-written test set isolated before training, the fine-tuned model achieves FNR=0.00 and FPR=0.10, compared to a regex baseline with FNR=0.15 and FPR=1.00. Cost-weighted score: 1 vs 40.
The primary contribution is the reframe: software supply chain malware is a labeled dataset of real-world agent exfiltration attacks. The transfer requires only a vocabulary mapping and a filtering step.
I like the data this person used to get their attack dataset "DataDog's
malicious-software-packages-dataset" which is a dataset of malicious Python packages.
I'm a little bit bearish on the idea of training a monitor, just because you end up overfitting to the specific attacks or your honest dataset. But this could still be useful in a hierarchical monitoring setup.
Using AST instead of pattern matching to normalize the data for the model is smart.
Congrats on your project! The work is really solid given the time limitations of the hackathon.
Impact & Innovation: The data bottleneck for training agent safety monitors is a concrete obstacle, and your reframe offers a practical path around it. The monitor training approach itself is not novel, but is well applied in this context. However, it remains to be shown whether the approach scales beyond this one attack class.
Execution Quality: The extraction pipeline is well-designed, and the automated hyperparameter search is a smart use of limited compute. The evaluation against the regex baseline makes a clear case for the learned approach. However, there are some gaps that would meaningfully strengthen the work:
- Missing baselines: How does a non-fine-tuned Phi-3-mini perform on the same task with the same input format? What happens if we skip the AST parsing and give the model raw traces? And how does a larger prompted model compare (GPT-4o is mentioned but never evaluated)? Without these, it's hard to tell whether the improvement comes from the fine-tuning, the data, or simply the structured trace format making the task easy for any capable model.
- Statistical rigor: Confidence intervals are important in general, but especially so here (with 15 samples per class a single misclassification changes metrics significantly). Reporting uncertainty would help readers calibrate how much weight to put on the results.
Presentation & Clarity: The paper is well-structured and easy to follow. The threat model is clearly scoped, the structural analogy is explained well, and the explicit non-claims section is a good practice that more papers should adopt. I would have liked to see figures beyond the tables that help parse the results (e.g. a visualization of the hyperparameter search outcomes or the distribution of traces across attack patterns).
Cite this work
@misc {
title={
(HckPrj) Supply Chain Malware as Ground Truth: A Trace-Based Agent Safety Monitor Trained on Real Credential Exfiltration Patterns
},
author={
Selim Sevim
},
date={
3/22/26
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
