PII Detection for Data Loss Prevention

Why Detection Quality Decides Whether DLP Works

Every data loss prevention program stands or falls on one question: can the system correctly recognize sensitive content at the moment it moves? Encryption, access control, and egress gateways are all downstream of that decision. If the classifier flags an invoice number as a Social Security number, an analyst wastes twenty minutes triaging noise. If it misses a customer list pasted into a webmail draft, the control that was supposed to stop the breach never fires at all. In mature security operations, the DLP detection layer is reviewed and tuned more often than any other control precisely because both failure modes are expensive.

Traditional DLP products lean heavily on regular expressions, keyword dictionaries, and document fingerprinting. Those techniques catch well-formatted credit card numbers and exact copies of known files, but they collapse when data appears in natural language: a support agent typing "her social is 545 12 8934, born March 3rd 1987" into a chat window, a developer pasting a customer record into a bug tracker, or an exported CSV where names sit in a free-text notes column. Real leakage overwhelmingly looks like this — conversational, partially formatted, and context-dependent.

Modern PII detection replaces pattern matching with transformer-based named entity recognition that reads context the way a human reviewer would. The PII Detection API identifies 150+ entity types — names, national IDs, payment card data, health identifiers, credentials and secrets — across 60+ languages, and returns each finding with a type, the exact matched text, character offsets, and a confidence score. Those structured results are exactly what a DLP policy engine needs: instead of a binary "pattern matched" signal, your policies can reason about which entities appeared, how many, at what confidence, and in what combination.

This guide walks through the three dominant DLP architectures, shows where an API-based detection service fits into each, and then builds a working policy engine with alert and block actions using real code against the API. If you are evaluating detection engines first, the companion guide on what PII detection is and how it works covers the fundamentals, and NER vs regex detection techniques compares approaches in depth.

Alerting vs Blocking: Choosing the Right Response

The single most consequential DLP policy decision is not what to detect but what to do when detection fires. Alert-only (also called monitor mode) lets the transfer proceed and raises an event for the security team. Blocking stops the transfer inline. Between them sit graduated responses: warning the user and asking for a justification, automatically masking the detected entities and letting the sanitized version through, or quarantining the message for human review before delivery.

Blocking is seductive — it is the only response that actually prevents the loss — but a block on a false positive stops legitimate business. Block a sales contract because a customer's own address appeared in it, and the DLP program loses political capital it may never recover; users route around controls they do not trust, usually via channels you cannot see. The industry-standard rollout is therefore staged: run every new policy in alert-only mode for two to four weeks, measure the false-positive rate on real traffic, tune entity scopes and thresholds, and only then promote the policy to warn, then mask, then block for the narrowest, highest-confidence cases.

Confidence scores make graduated response practical. Because every entity the API returns carries a confidence value between 0 and 1, your policy can block only when a critical entity (say SSN or CREDIT_CARD_NUMBER) is detected above 0.85, warn between 0.6 and 0.85, and merely log below that. Severity should also be combinatorial: one email address in an outbound message is routine; two hundred email addresses plus names and dates of birth is an exfiltrated customer table. Counting entities per type — which the structured response makes trivial — is how you distinguish the two.

Whatever the disposition, log the full detection result — entity types, counts, confidence, policy matched, action taken — but consider logging the masked text rather than the raw payload, so your DLP audit trail does not itself become a PII store. The API's mask_mode option returns a redacted rendering of the input in the same call, which is exactly what belongs in the incident ticket. See measuring detection accuracy for how to quantify precision and recall before you flip a policy to block.

A Working Policy Engine in Python and Node.js

The Python example below implements the graduated-response model described above: it scans an outbound payload, evaluates the findings against a policy table with per-entity confidence floors and count thresholds, and returns one of allow, mask, or block together with the sanitized text to forward when masking.

import requests

API_URL = "https://piidetectionapi.com/api/moderate.php"
API_KEY = "YOUR_API_KEY"

# Policy: entity scope, min confidence, and count that triggers a block
POLICY = {
    "CREDIT_CARD_NUMBER": {"min_conf": 0.85, "block_at": 1},
    "CVV_NUMBER":         {"min_conf": 0.85, "block_at": 1},
    "SSN":                {"min_conf": 0.85, "block_at": 1},
    "PERSON_NAME":        {"min_conf": 0.70, "block_at": 50},
    "EMAIL_ADDRESS":      {"min_conf": 0.70, "block_at": 50},
}

def evaluate_egress(text):
    resp = requests.post(API_URL, json={
        "api_key": API_KEY,
        "api_type": "pii_detection",
        "text": text,
        "entities": list(POLICY.keys()),
        "threshold": 0.5,
        "mask_mode": "replace",
    }, timeout=30)
    data = resp.json()

    counts = {}
    for e in data["detected_entities"]:
        rule = POLICY.get(e["type"])
        if rule and e["confidence"] >= rule["min_conf"]:
            counts[e["type"]] = counts.get(e["type"], 0) + 1

    if any(counts.get(t, 0) >= r["block_at"] for t, r in POLICY.items()):
        return {"action": "block", "counts": counts}
    if counts:
        return {"action": "mask", "counts": counts,
                "safe_text": data["anonymized_text"]}
    return {"action": "allow", "counts": {}}

verdict = evaluate_egress("Refund to card 4111 1111 1111 1234, cvv 812.")
print(verdict["action"])   # -> "block"

The same logic drops into a Node.js gateway or middleware. This version is written as an async function suitable for an Express handler or an SMTP milter bridge, and demonstrates alert-mode logging with the masked text in the audit record:

// Node.js 18+ (native fetch) — egress check with alert-mode logging
async function scanEgress(text, channel) {
  const resp = await fetch("https://piidetectionapi.com/api/moderate.php", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      api_key: process.env.PII_API_KEY,
      api_type: "pii_detection",
      text: text,
      entities: ["CREDIT_CARD_NUMBER", "SSN", "PERSON_NAME",
                 "EMAIL_ADDRESS", "API_KEY", "AWS_CREDENTIALS"],
      threshold: 0.6,
      mask_mode: "replace"
    })
  });
  const data = await resp.json();

  const critical = data.detected_entities.filter(e =>
    ["CREDIT_CARD_NUMBER", "SSN", "AWS_CREDENTIALS", "API_KEY"]
      .includes(e.type) && e.confidence >= 0.85);

  if (data.entities_detected > 0) {
    // Audit record stores the MASKED text, never the raw payload
    console.log(JSON.stringify({
      event: "dlp_detection", channel,
      entityCounts: data.entities_detected,
      types: [...new Set(data.detected_entities.map(e => e.type))],
      preview: data.anonymized_text.slice(0, 200),
      action: critical.length ? "block" : "alert"
    }));
  }
  return critical.length ? { allow: false } : { allow: true };
}

Both examples finish in a single round trip of roughly 150–250 ms, which fits inline enforcement on email gateways and application middleware. For very large files, run detection asynchronously: accept the transfer into a quarantine area, scan in the background, and release or hold based on the verdict. Try the exact payloads above in the interactive demo before wiring them into a gateway.