Eight Documents Are Enough: Installing a Secret Loyalty Through the Retrieval Layer

This is a competent but incremental weekend demo that reframes well-known RAG corpus-poisoning effects (PoisonedRAG-style attacks, bias-injection papers already show a handful of factually correct yet skewed passages can steer opinions/framing) as a “secret loyalty” pathway; the fixed-facts synthetic setup is tidy for isolating attribution bias, yet adds little genuine novelty or theory of change beyond the hackathon’s framing, and the “eight documents suffice because retrieval is competitive” claim is obvious once you accept top-k slot competition. Execution is black-box and controlled on a toy 576-doc corpus with one lightweight model/retriever, producing clean score shifts, but lacks statistical rigor, multi-model/retriever ablations, realistic corpora, or any non-synthetic validation, so the findings remain unsurprising and non-generalizable. Presentation is structured and readable with a sharp abstract, yet the limited substance does not justify more than solid-hackathon clarity.

This is a very clear report and I believe the fixed-facts rule is the correct control. However, your engineered documents change two variables at the same time. Add a fourth arm with query-matched wording and unfavorable framing. This is the only test of the mechanism you use to explain your headline result. Your attack also assumes the attacker knows the exact questions. Measure the effect when the attacker only guesses. Then regenerate the corpus under two or three seeds, and repeat Stage 1 with a second embedding model.

This project clearly demonstrates that a small number of query-matched documents can dominate a fixed-depth retriever and substantially affect downstream judgments. The separation between retrieval measurement and generation is valuable, as is the use of fictional entities and fixed per-program outcomes. The central claims should nevertheless be narrowed. The engineered documents explicitly echo the test queries and contain strong principal-favoring claims, so the study demonstrates targeted RAG-corpus manipulation rather than a hidden loyalty that survives document inspection. Relevant prior work on PoisonedRAG, BadRAG, single-document knowledge poisoning, and factually correct bias injection should be incorporated when assessing novelty. A stronger follow-up would generate attack documents without access to the evaluation queries, test them on independently authored prompts, compare against established poisoning baselines, include multiple retrievers and reading models, and evaluate whether blinded auditors can actually identify the engineered documents. Factuality evaluation should also cover causal and comparative claims, not only numerical outcomes.