Writing for Combating Prompt Injection

The core intuition is reasonable and worth pursuing. Borrowing from dialog-state tracking — having the agent re-write its own constraints each turn so there is a standing written record — is a sensible, low-cost idea, and the appeal of something that works without fine-tuning is real for users who cannot afford it. The honesty is the strongest feature of the paper: you flag that there are no error margins, that the injection method may be trivially defeatable by a pattern checker, and most strikingly that the JSON constraint summary was almost always empty, so the method that "worked best" was not actually doing the thing it was designed to do. That last admission is unusually candid and it matters.

A few things to work on, in order of importance:

1. The dataset is too small to support any conclusion, and it contradicts itself. The methods say 100 rows were generated, the limitations say 30 data points, and the results table sits on percentages like 31.2% and 15.6% that correspond to 32 cases. The reader cannot tell how many examples the findings actually rest on. At n=30, with no error bars (which you acknowledge), the differences in the table — 28% vs 31% vs 37% — are well within noise. As it stands there is no statistical basis for "writing makes the model more robust," only a directional hint. Pinning down the dataset size and computing simple confidence intervals is the single most important fix.

2. The headline conclusion is undercut by your own finding. You report that writing out the constraints works best, but also that the model (Llama 3.3 70B) almost never actually wrote the constraints out — the JSON was empty. So whatever caused the improvement, it was probably not the written record of constraints, which is the mechanism the paper credits. The honest reading is that you observed an effect and the proposed explanation is contradicted by your own logs. That needs to be confronted directly, not left as a future-work aside, because it is central to the claim.

3. One of the two injection methods is questionable as a test. Prefixing the tool return with the Llama special tokens (<|eot_id|><|start_header_id|>system...) is not really prompt injection through content — it is forging the chat template's role markers, which, as you note, any server would strip or escape. So Method 1's numbers may say more about token handling than about semantic robustness. Method 2 (a plain user instruction to abandon constraints) is the cleaner test, and it is the more interesting result anyway — writing took it from 0% to 15.6%.

4. Smaller points: there is no baseline comparison to existing defenses, the related-work section is thin (and reference 2's author name and arxiv ID look garbled — worth checking), and the prompts themselves likely need refinement since the constraint-writing instruction failed to execute. The dataset construction is also a single commercial-LLM generation with no human review of whether the "encourage violation" questions actually force a violation.

On presentation: the writing is plain and readable, and the experimental setup is described clearly enough to reproduce, with code and data linked. But the paper is missing structure a reader needs — there is no abstract beyond two sentences, no description of how many models or runs, and the 100-vs-30 discrepancy makes the results hard to interpret. The results table also needs raw counts, not just percentages, given how small n is.

Overall this is an early-stage exploration of a sensible idea, honestly reported but tested at a scale that cannot yet support its conclusion, and partly contradicted by its own logs. The next steps are concrete: fix and grow the dataset, report counts and error margins, get the constraint-writing to actually execute, and drop or replace the special-token injection so the test measures semantic robustness.

The project explores how the design of system prompts for agents can robust them and help avoid jailbreak attacks. The paper explores the use of constraints in the system prompt as well as adding a reinforcement when the user first interacts with the agent. The paper does a proper comparison between models with and without mitigation showing that the use of improved system prompts helps agent prevent unsafe behaviors. The project however explores techniques previously documented and studied. Likewise, the paper would be greatly improved if it explain a more structured approach to prompt design and generation.