Retrieval-augmented generation moves your documents into embeddings, vector databases, and third-party model prompts. This guide maps every point where personal data leaks out of a RAG stack — and shows how to close each one with scan-before-embed, context scrubbing, and query filtering.
Start ReadingRetrieval-augmented generation is the pattern that made enterprise LLMs useful: instead of relying on a model's frozen training data, you embed your own documents into vectors, store them in a vector database, retrieve the most relevant chunks for each question, and hand them to the model as context. The architecture is powerful precisely because it moves your data through the system — and that is also why it multiplies privacy risk rather than merely inheriting it.
Consider what actually happens when a team indexes "the knowledge base." The knowledge base is rarely just polished documentation. It is support ticket archives with customer emails and phone numbers, HR wikis with employee records, contracts with signatories and bank details, meeting transcripts with who-said-what, and CRM exports with every field the sales team ever collected. RAG ingestion pipelines copy all of it into a new store — the vector database — that sits outside every access-control and retention regime the source systems had.
The result is a triple exposure. First, PII now exists in an extra copy (chunks stored alongside vectors) that your data map probably doesn't know about. Second, PII flows to third parties twice per query: to the embedding provider at ingestion and to the LLM provider at generation. Third, retrieval is probabilistic: a semantic search for "customers unhappy about billing" can surface a chunk containing a specific person's card details to any employee — or any end user — who can type a query. Access control by similarity score is no access control at all.
The good news is that RAG's modularity gives you clean interception points. Text is a string at every stage boundary, and a detection API that returns entities with offsets — like the one documented on the API overview — can sit at each boundary as a filter. The rest of this guide walks the boundaries one by one. For the adjacent problem of guarding prompts and completions generally, see PII detection for LLM guardrails.
Trace a document from source system to generated answer and PII has six distinct opportunities to escape. Each stage has a different failure mode and a different control.
Loaders pull documents from wikis, drives, mailboxes, and ticket systems — often with service-account permissions broader than any human's. PII enters here wholesale, including from files nobody remembered were in the folder. This is the cheapest place to stop it: one scan per document, before anything downstream exists. See document PII scanning for the extraction side.
Splitters cut documents into 300–1,000-token chunks and duplicate PII into overlap windows — a name in a 100-token overlap now lives in two chunks. Chunking also strips context: "the patient" in chunk 12 and "Jane Morrow, DOB 4/2/1979" in chunk 11 look unrelated to a chunk-level scanner. Detect on the full document first, then chunk the masked output, and offsets stay coherent.
Every chunk is POSTed to an embedding model — frequently a third-party API. If the chunk contains an SSN, that SSN has now been disclosed to another processor, logged in their infrastructure, and transformed into a vector you cannot easily audit. Embedding raw PII is the moment the toothpaste leaves the tube; everything after is remediation.
Vector stores keep the original chunk text as metadata alongside each vector — that is what gets stuffed into prompts later. So the "vector database" is really a plaintext PII database with a similarity index, typically without field-level encryption, row-level ACLs, or retention rules. Deleting one person's data for a GDPR erasure request means finding their entities across millions of opaque chunks.
Top-k similarity search returns whatever is semantically closest, with no notion of need-to-know. A support agent asking "how do refunds work?" can retrieve a chunk quoting a specific customer's refund, name, and card tail. Retrieval is where PII crosses from storage into active use — the last practical checkpoint before it reaches the model.
The assembled prompt — system message, retrieved chunks, user question — goes to the LLM provider, and the completion goes back to the user, into logs, and often into evaluation datasets. A model can also echo retrieved PII into an answer seen by someone who should never see it. Output-side checks belong to real-time chatbot filtering; in RAG, the goal is that PII never gets this far.
A tempting assumption is that embedding is anonymization — after all, a 1,536-dimensional float vector looks nothing like "Jane Morrow, SSN 512-84-1177." Regulators and researchers disagree, for three reasons that every RAG architect should be able to recite.
First, embeddings are invertible in practice. Published inversion attacks reconstruct substantial portions of input text from embeddings alone — recovering names, and in some studies the majority of exact tokens — because embeddings are engineered to preserve meaning. Under GDPR's "means reasonably likely to be used" test, data that can be reconstructed is personal data; vectors derived from PII are therefore best treated as pseudonymized at most, never anonymized. Second, the chunk text travels with the vector anyway. Inversion is an academic worry; the metadata field storing the original chunk in plaintext is an operational certainty.
Third, data-subject rights don't stop at the index. If a person exercises erasure or access rights, vectors and chunks derived from their data are in scope. Without detection at ingestion you cannot even answer "which chunks contain this person?" — the store is organized by semantic similarity, not by data subject. Teams that scanned at ingestion can answer from their scan logs; teams that didn't face re-scanning the entire corpus under a regulatory clock. The same logic applies to HIPAA: chunks derived from PHI keep their PHI status, with everything the HIPAA PHI guide implies about BAAs with every provider in the chain.
The architectural conclusion is stark but freeing: the only clean vector database is one that never ingested PII. Mask before embedding and the compliance questions mostly evaporate — vectors derived from "[NAME], [SSN]" are derived from no one. That is the scan-before-embed pattern, and it costs one API call per document.
One table to pin above the architecture diagram. Each RAG stage pairs with a detection control and a mask_mode choice; the API contract for all of them is a single POST described in the API documentation.
| Pipeline Stage | Primary Risk | Detection Control | Recommended mask_mode |
|---|---|---|---|
| Ingestion | Bulk PII enters from uncurated sources | Scan every document before any downstream processing; quarantine or mask on hit | hash — preserves entity linkage for retrieval quality |
| Chunking | PII duplicated into overlaps; context split across chunks | Detect on full document, chunk the masked text (stay under the 50,000-char request limit per call) | Inherited from ingestion scan |
| Embedding | Disclosure to embedding provider; invertible vectors | Hard gate: refuse to embed any chunk that has not passed the scan | hash or replace |
| Vector DB | Unmanaged plaintext PII copy; erasure requests unanswerable | Periodic audit scans of stored chunk metadata; keep scan logs as data map | redact for legacy cleanup |
| Retrieval | Similarity search bypasses need-to-know | Scrub retrieved context before prompt assembly (defense-in-depth for pre-masking gaps) | replace — placeholders keep prompts readable |
| Query | Users paste PII into questions; it lands in logs and providers | Filter user queries synchronously before embedding/search | replace, low threshold |
| Generation | Model echoes residual PII to unauthorized viewers | Output scan as final guardrail (see LLM guardrails) | redact |
The scan-before-embed pattern inserts one step between document loading and chunking: POST the document text to the detection endpoint, take the anonymized_text from the response, and let only that masked text proceed to chunking and embedding. Everything else in your RAG stack — splitters, embedding model, vector store, retriever — stays untouched, which is why the pattern retrofits cleanly onto LangChain, LlamaIndex, or hand-rolled pipelines alike.
Three implementation details matter. Scan whole documents, not chunks. Detection accuracy rises with context — the model that sees "attending physician Dr. Chen noted the patient, Jane Morrow, ..." makes better decisions than one seeing a 300-token fragment. It is also cheaper: one API call per document instead of one per chunk. For documents beyond the 50,000-character request limit, split on natural boundaries with modest overlap and merge results, as covered in the document scanning guide.
Choose your entity list deliberately. A customer-support knowledge base wants customer identifiers gone (PERSON_NAME, EMAIL_ADDRESS, PHONE_NUMBER, SSN, CREDIT_CARD_NUMBER, ADDRESS) but should keep product names and URLs intact. An HR assistant needs the special-category types too. Use exclude_entities and custom_instruction to protect domain terms — ticket IDs, SKUs, internal hostnames — from over-masking that would degrade retrieval.
Record what you found. The structured detected_entities array — types, counts, offsets, confidence — is your vector store's data map. Persist it per document (not the matched text itself) and you can answer auditors' and data subjects' questions later without re-scanning the corpus. Teams shipping SaaS AI features treat these scan records as first-class pipeline artifacts, versioned alongside the embedding model name.
Three building blocks: a raw cURL call to see the contract, a Python scan-before-embed ingestion function, and a LangChain-style integration that masks documents and retrieved context in one place.
# cURL — scan a document before embedding, hash mode for consistent entity linking
curl -X POST https://piidetectionapi.com/api/moderate.php \
-H "Content-Type: application/json" \
-d '{
"api_key": "YOUR_API_KEY",
"api_type": "pii_detection",
"text": "Ticket #4412: Jane Morrow ([email protected], +1 415-555-0182) reports a duplicate charge on card ending 4242.",
"entities": ["PERSON_NAME","EMAIL_ADDRESS","PHONE_NUMBER","CREDIT_CARD_NUMBER","SSN","ADDRESS"],
"mask_mode": "hash",
"threshold": 0.45
}'
# Python — scan-before-embed ingestion: only masked text ever reaches the embedder
import requests
PII_ENDPOINT = "https://piidetectionapi.com/api/moderate.php"
RAG_ENTITIES = ["PERSON_NAME", "EMAIL_ADDRESS", "PHONE_NUMBER", "SSN",
"CREDIT_CARD_NUMBER", "ADDRESS", "DATE_OF_BIRTH",
"FINANCIAL_ACCOUNT_NUMBER", "API_KEY", "PASSWORD"]
def mask_document(text: str) -> tuple[str, list]:
resp = requests.post(PII_ENDPOINT, json={
"api_key": "YOUR_API_KEY",
"api_type": "pii_detection",
"text": text, # <= 50,000 chars per request
"entities": RAG_ENTITIES,
"mask_mode": "hash", # same person -> same token across chunks
"threshold": 0.45,
"custom_instruction": "Do not mask ticket IDs like #4412 or product SKUs",
}, timeout=30)
resp.raise_for_status()
data = resp.json()
return data["anonymized_text"], data["detected_entities"]
def ingest(doc_id: str, raw_text: str, splitter, embedder, vector_store, audit_log):
masked, entities = mask_document(raw_text)
audit_log.write(doc_id, [{"type": e["type"], "start": e["start"],
"end": e["end"]} for e in entities])
chunks = splitter.split_text(masked) # chunk AFTER masking
vectors = embedder.embed_documents(chunks) # embedder never sees raw PII
vector_store.add(doc_id, chunks, vectors)
# Python — LangChain-style integration: a transformer for ingestion plus a
# retriever wrapper that scrubs context before it reaches the prompt
class PIIMaskingTransformer:
"""Drop-in document transformer: run over loader output before the splitter."""
def transform_documents(self, documents):
for doc in documents:
doc.page_content, entities = mask_document(doc.page_content)
doc.metadata["pii_entities_removed"] = len(entities)
return documents
class ScrubbedRetriever:
"""Wraps any retriever; defense-in-depth for legacy/unmasked indexes."""
def __init__(self, base_retriever):
self.base = base_retriever
def get_relevant_documents(self, query):
safe_query, _ = mask_document(query) # filter the user's question too
docs = self.base.get_relevant_documents(safe_query)
for doc in docs: # scrub retrieved chunks pre-prompt
doc.page_content, _ = mask_document(doc.page_content)
return docs
The same two primitives port directly to LlamaIndex node post-processors or a custom pipeline. Full request and response schemas are in the API documentation; start with a free key via get started.
If every document were perfectly masked at ingestion, retrieval-time scrubbing would be redundant. Real systems are messier: indexes built before the masking step existed, teams that bulk-loaded a drive "temporarily," connectors that bypass the pipeline, and detection itself — which, like all statistical systems, has a recall short of 100%. Scrubbing retrieved context is the second gate that turns any single miss into a non-event.
Mechanically it is simple: after the retriever returns its top-k chunks and before prompt assembly, concatenate the chunks (or send them individually) to the detection endpoint with mask_mode: "replace" and rebuild the context from anonymized_text. Placeholders like [NAME] and [EMAIL] keep the passage grammatical, so the LLM still reasons over it correctly — an answer that says "contact [NAME] in billing" is exactly the leak-free behavior you want, and it signals to users that a redaction occurred rather than silently dropping facts.
Retrieval-time scrubbing is also the right home for audience-dependent policy. Ingestion-time masking is one-size-fits-all, but at query time you know who is asking: an internal compliance analyst tool might scrub only SSN and CREDIT_CARD_NUMBER, while a customer-facing assistant scrubs every identity type at a low threshold. Because the entity list is just a request parameter, the same index serves both audiences with different entities arrays — no re-indexing required.
Budget for it honestly: it adds one detection call per query on the retrieval path. With typical processing times of a few hundred milliseconds for prompt-sized payloads (see the latency section below), the pattern fits comfortably inside interactive budgets, and the chunks can be scanned in a single call since top-k context rarely approaches the 50,000-character limit.
The query box is the leak point teams forget, because the data flows the "wrong" way — from the user into the pipeline. Users paste whole email threads and ask "summarize this." They ask "why was Jane Morrow's claim at 14 Elm Street denied?" Each such query is PII, and it fans out further than any document: it is embedded (one provider), searched against the index, sent to the LLM inside the prompt (another provider), and written to query logs, traces, and analytics — the exact surfaces covered in log file PII scanning.
A synchronous filter fixes this at the front door: scan the query with a low threshold before anything else touches it, and proceed with the masked form. Retrieval quality usually survives, and often improves — embeddings of "why was [NAME]'s claim at [ADDRESS] denied" still land in the claims-denial neighborhood of vector space, which is where the useful documents live. When an application genuinely needs the identifier (a lookup tool, say), route that path through structured search with proper authorization instead of letting it ride the semantic index.
Query filtering earns a second dividend: clean telemetry. RAG teams replay production queries to build evaluation sets, tune retrievers, and fine-tune models. If raw queries contain PII, every one of those downstream uses inherits it — a masked query stream makes replay corpora, dashboards, and fine-tuning datasets safe by construction. The chatbot PII filtering guide covers the same pattern for conversational front-ends, including streaming considerations.
Configuration guidance: run mask_mode: "replace" with threshold around 0.35–0.45 on queries. Queries are short, so context is thin and confidence runs lower than on documents; a lenient threshold buys recall where it is cheap, because over-masking a query costs a slightly vaguer search, not a lost fact.
The API offers three masking strategies, and RAG is the use case where the choice matters most — because masked text is not just stored, it is searched and reasoned over.
Substitutes typed placeholders: "Contact [NAME] at [EMAIL]." Best for retrieved context and query filtering, where a human or an LLM reads the text next and should see that something was removed and what kind of thing it was. The default mode, and the right one everywhere legibility beats linkability.
Replaces each unique value with a stable hash token: every occurrence of the same email becomes the same token, across chunks and across documents. This is the RAG ingestion workhorse. Retrieval still connects "the customer in chunk 3" with "the customer in chunk 40"; multi-document reasoning about the same (unnamed) person keeps working; and re-scanning a re-ingested document produces identical chunks, so vectors stay stable and deduplicable.
Deletes the matched span outright. Strongest guarantee, least utility: sentences lose their shape ("Contact at about the charge"), which hurts both embedding quality and LLM comprehension. Reserve it for legacy vector-store cleanup, generation-output hard gates, and compliance exports where nothing recoverable may remain — and prefer replace anywhere the text will be read again.
Evaluate a RAG privacy layer on two axes at once, because they pull against each other. The privacy axis is recall: of the identifiers present in a labeled sample of your corpus, how many did the ingestion scan catch? Build the test set from your own documents — a few hundred labeled chunks spanning every source system — and measure per entity type, since an aggregate score hides the entity family you are worst at. The methodology, including how to label and how to compute per-type precision, recall, and F1, is in measuring PII detection accuracy.
The utility axis is retrieval quality: does masking change which chunks are retrieved and how good the answers are? Run your standard RAG evaluation (retrieval hit-rate, answer faithfulness) on masked and unmasked twins of the same corpus. In practice, hash-mode masking moves retrieval metrics only slightly — questions are about topics, and topics survive masking — but measuring it convinces stakeholders and catches over-masking of domain terms early, which is exactly what custom_instruction exists to fix.
On latency: ingestion-time scanning is free in any practical sense, since indexing is asynchronous batch work — a few hundred milliseconds per document disappears inside a pipeline that is already calling an embedding API per chunk. The query path is where budgets bind. A query filter adds one call (~150–300 ms for short text) and retrieval scrubbing adds one more; run them where they overlap naturally — the query scan can proceed in parallel with nothing, but the context scrub can overlap prompt assembly — and total added latency typically lands under half a second, well within interactive tolerance for assistants that already spend seconds generating.
If the budget is tighter, spend recall where it is cheap: keep the strict, low-threshold scan at ingestion (offline) and run the query-path checks at moderate thresholds. And before committing, benchmark with your own payload sizes against the live endpoint — get started issues a key in minutes, and pricing scales by volume, so a proof-of-concept on a real corpus slice is an afternoon's work.
exclude_entities and custom_instruction, and catch by running your retrieval evals on masked vs unmasked twins of a sample corpus.ScrubbedRetriever pattern above) so nothing sensitive reaches prompts while the index is dirty. Then remediate: export chunk metadata, scan it in batches, and either rewrite chunks with masked text (keeping vectors, accepting slight text/vector drift) or re-embed the masked chunks properly. The scan output doubles as your data map — you finally learn which sources contaminated the index, which tells you which connectors need the ingestion gate first.redact mode. The output side, including streaming and latency tactics, is covered in PII detection for LLM guardrails.Paste a knowledge-base chunk into the live demo and watch every identifier surface with offsets and confidence — then wire one POST call into your ingestion pipeline.
Try the Live Demo View Pricing