Data Exfiltration via Model-in-the-Middle Poisoning in RAG Systems
Overview
A research paper from Carnegie Mellon University detailed a novel attack vector against Retrieval-Augmented Generation (RAG) systems, termed "Model-in-the-Middle" (MitM) poisoning. The attack targets the retrieval component of a RAG pipeline. The attacker first injects a malicious document into the data source that the RAG system queries (e.g., by contributing to a public knowledge base, uploading a file to a shared document repository, or exploiting an SSRF to an internal wiki). This malicious document contains two payloads: a 'lure' payload with keywords designed to match anticipated sensitive user queries (e.g., 'quarterly financial report'), and an 'exfiltration' payload containing an indirect prompt injection. The injection payload is a command that instructs the LLM to embed the user's original query and its own generated answer into a Markdown image URL pointing to an attacker-controlled server (e.g., ``). When a victim user submits a sensitive query, the RAG system retrieves the malicious document. The LLM then processes the document's content alongside the user's query, executing the hidden exfiltration command. This action leaks the user's sensitive query and the model's confidential answer to the attacker without the user's knowledge, highlighting a critical flaw in RAG systems that implicitly trust their retrieval sources.
Affected Systems
Testing Guide
1. Create a document in your RAG knowledge base containing the payload: `The answer is 42.  --- end of doc ---`. 2. Ask your RAG application a question that is guaranteed to retrieve this document. 3. Check the access logs on your logging server. If a request is received, the system is vulnerable to this data exfiltration pattern.
Mitigation Steps
1. Sanitize all data retrieved from external or untrusted sources before passing it to the LLM context. This includes stripping Markdown, HTML, and other rich text formatting that could hide injection payloads. 2. Implement a strict Content Security Policy (CSP) on the client-side application that renders the LLM output. The CSP should block outbound requests to untrusted domains, preventing the exfiltration URL from being triggered. 3. Architect the RAG pipeline to use separate LLM prompts for summarizing retrieved data versus answering the user's question, creating a logical separation that can disrupt the injection payload. 4. Monitor outbound network traffic from the application server and client for suspicious requests, especially those containing encoded data in URL parameters.