PII Detection for Government & FOIA Processing

Transparency and Privacy Are Both the Law

Government agencies live under a double mandate. Freedom of Information laws — federal FOIA and its state public-records counterparts — require them to disclose records to anyone who asks. The Privacy Act of 1974, FOIA's own Exemptions 6 and 7(C), and a lattice of confidentiality statutes simultaneously require them to withhold personal information about the individuals named in those records. Every released page is a judgment call between the public's right to know and a citizen's right to be let alone.

The volume makes that judgment call an industrial process. Federal agencies received well over a million FOIA requests in a single recent fiscal year, and backlogs have grown even as staffing has not. A single request can sweep in tens of thousands of pages of emails, case files, and reports — each of which a FOIA analyst must read line by line hunting for names, SSNs, home addresses, phone numbers, medical details, and other material that must be redacted before release under Exemption 6 or 7(C).

The PII Detection API automates the hunting. Submit record text to a single endpoint and get back every personal identifier with its entity type, exact character offsets, and a confidence score, plus an optional masked rendering. Analysts stop scanning for identifiers and start doing the work only they can do: weighing the privacy interest against the public interest, applying the foreseeable-harm standard, and documenting the withholding. New to the technology? Start with our complete guide to PII detection.

Where Personal Data Hides in Agency Records

FOIA review sets are the obvious surface: responsive emails carry signature blocks, cc lists, and personal anecdotes; investigative files name complainants, witnesses, and low-level employees whose identities Exemption 7(C) protects; personnel-adjacent records trigger Exemption 6 nearly line by line.

Citizen correspondence — constituent letters, benefit applications, complaint hotlines, 311 tickets — arrives saturated with self-disclosed PII: SSNs volunteered "to help you find my file," children's names, medical histories, immigration details. These flow into CRMs, ticketing systems, and shared inboxes that were never designed as systems of records.

Open-data programs publish datasets scraped from operational systems — permit registries, inspection reports, spending ledgers — where free-text description fields routinely smuggle names and addresses into "non-personal" data. And records-management migrations move decades of legacy documents into new repositories without anyone knowing which boxes contain SSNs from an era when they were printed on everything.

Segregability, computed

FOIA requires releasing all reasonably segregable non-exempt material. Entity-level offsets are segregability made computable: the API tells you precisely which characters carry the privacy interest, so you redact those spans and release everything else — the exact posture FOIA officers must defend in litigation.

The Legal Framework for Government PII

Four bodies of law converge on the same operational requirement: know exactly where personal identifiers sit in every record you hold or release.

FOIA Exemptions 6 & 7(C)

Exemption 6 protects personnel, medical, and similar files whose disclosure would constitute a clearly unwarranted invasion of personal privacy; Exemption 7(C) applies a lower "could reasonably be expected to" bar to law-enforcement records. Both require the agency to first find the personal information — names, identifying details, contact data — before balancing privacy against public interest. Automated detection is that finding step, applied uniformly across every page.

Privacy Act of 1974

The Privacy Act restricts disclosure of records retrieved by an individual's name or identifier, mandates accounting of disclosures, and gives individuals civil remedies for violations. Knowing which documents and fields contain which identifiers is a precondition for honoring access and amendment requests, enforcing routine-use limits, and reporting incidents involving systems of records.

Open Data & OMB Mandates

The OPEN Government Data Act and OMB guidance push agencies to publish machine-readable data by default — while explicitly requiring a mosaic-effect privacy analysis before release. Scanning free-text columns for residual identifiers before a dataset reaches the portal is the concrete control auditors look for, and the one most often missing.

Records Management & Breach Rules

NARA schedules govern retention; OMB M-17-12 and state analogs govern breach response. Both depend on a PII inventory: you cannot apply a privacy-driven retention rule, or scope a breach notification, without knowing which records contain which identifiers. Detection at ingestion builds that inventory as a by-product of normal processing.

The Cost of Getting the Balance Wrong

Errors in either direction are expensive. Over-redaction draws FOIA litigation, court-ordered releases, and attorney-fee awards — and post-2016 FOIA amendments codified the foreseeable-harm standard precisely to curb reflexive withholding. Under-redaction is worse: agencies have accidentally released SSN-laden spreadsheets, exposed informants' identities, and published open-data files that re-identified crime victims. Those incidents trigger Privacy Act liability, congressional attention, and lasting damage to citizens who never consented to appear in a database.

Manual review does not scale out of this dilemma; consistency does. An automated first pass applies the same detection standard to page one and page ninety thousand, leaves an auditable record of what was found, and frees analysts to spend their expertise on the balancing test rather than the treasure hunt.

Consistency also compounds across offices. When every component of an agency runs the same detection configuration, the FOIA office, the privacy office, and the open-data team stop arguing about what counts as an identifier and start from a shared, machine-generated inventory. Annual Chief FOIA Officer reports gain concrete numbers — records scanned, identifiers found, average time to first response — instead of estimates, and oversight questions get answered from logs rather than recollection.

Government Data Types We Detect

Entity coverage for the identifiers that appear in agency records, requests, and releases

Citizen & Employee Names

PERSON_NAME

SSNs & Taxpayer IDs

SSN, TAX_ID, NATIONAL_ID

Home Addresses

ADDRESS, CITY, ZIP_CODE

Contact Details

PHONE_NUMBER, EMAIL_ADDRESS

Birth Dates & Ages

DATE_OF_BIRTH, AGE

Government Document IDs

PASSPORT_NUMBER, DRIVERS_LICENSE_NUMBER

Medical & Benefits Data

MEDICAL_DATA, HEALTH_INSURANCE_ID

Financial Accounts

FINANCIAL_ACCOUNT_NUMBER, ROUTING_NUMBER

Digital Identifiers

IP_ADDRESS, DEVICE_ID, URL

Location Data

GPS_COORDINATES

Sensitive Categories

RELIGION, ETHNIC_GROUP, POLITICAL_AFFILIATION

Employment Details

EMPLOYMENT, MARITAL_STATUS

Mapping Legal Categories to Detection Entities

Privacy exemptions are written in legal language; detection requests are written in entity types. The table below translates the most common withholding categories into the entities arrays a FOIA or open-data pipeline would use. The complete list of 150+ supported types is on the entities page.

Detection flags the identifier; the analyst applies the balancing test. That division of labor is what makes the workflow both fast and legally defensible — no exemption decision is delegated to the model.

State and local governments can apply the same mapping to their own public-records statutes: the exemption labels change, but the identifier categories — names, SSNs, addresses, medical details, account numbers — are nearly universal across all fifty states' sunshine laws.

Legal Category	Authority	API Entity Types	Typical Action
Personal privacy in general records	FOIA Exemption 6	`PERSON_NAME`, `ADDRESS`, `DATE_OF_BIRTH`, `PHONE_NUMBER`, `EMAIL_ADDRESS`	Flag for analyst balancing; redact spans on withholding
Privacy in law-enforcement records	FOIA Exemption 7(C)	`PERSON_NAME`, `ADDRESS`, `DRIVERS_LICENSE_NUMBER`, `AGE`	Redact third-party identities (witnesses, agents, suspects not charged)
System-of-records identifiers	Privacy Act of 1974	`SSN`, `NATIONAL_ID`, `TAX_ID`, `MEDICAL_RECORD_NUMBER`	Inventory, restrict disclosure, log accounting
Medical and benefits information	Exemption 6; HIPAA where applicable	`MEDICAL_DATA`, `DIAGNOSIS`, `HEALTH_INSURANCE_ID`	Redact in full absent overriding public interest
Financial account details	Exemptions 4 & 6; state analogs	`FINANCIAL_ACCOUNT_NUMBER`, `CREDIT_CARD_NUMBER`, `ROUTING_NUMBER`	Redact in full
Open-data residual identifiers	OPEN Government Data Act; OMB guidance	All identifier types + `GPS_COORDINATES`, `IP_ADDRESS`	Zero-entity gate before portal publication

After Masking

Grant file: PI [PERSON_NAME], [EMAIL_ADDRESS], passport [PASSPORT_NUMBER], award disbursed to acct [FINANCIAL_ACCOUNT_NUMBER].

1M+

Annual Federal FOIA Requests

150+

Entity Types Detected

90%

Less Time Hunting Identifiers

60+

Languages for Multilingual Constituents

Integrate Detection Into Agency Workflows

One JSON endpoint behind your FOIA platform, CRM, or data portal. Full reference in the API documentation and API overview.

cURL — Scan a Responsive Record

curl -X POST https://piidetectionapi.com/api/moderate.php \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_API_KEY",
    "api_type": "pii_detection",
    "text": "Complaint filed by Teresa Vann, 4419 Loomis St, Dayton OH 45417, SSN 288-54-1173, regarding denial of benefits. Case officer: J. Whitcomb.",
    "entities": ["PERSON_NAME", "ADDRESS", "ZIP_CODE", "SSN", "PHONE_NUMBER", "EMAIL_ADDRESS"],
    "mask_mode": "replace",
    "threshold": 0.4
  }'

Python — FOIA Queue Processor with Exemption Tagging

import requests

API_URL = "https://piidetectionapi.com/api/moderate.php"

# Entity types that typically implicate Exemption 6 / 7(C)
EXEMPTION_MAP = {
    "SSN": "(b)(6)", "DATE_OF_BIRTH": "(b)(6)",
    "ADDRESS": "(b)(6)", "PHONE_NUMBER": "(b)(6)",
    "EMAIL_ADDRESS": "(b)(6)", "MEDICAL_DATA": "(b)(6)",
    "PERSON_NAME": "(b)(6)/(b)(7)(C)",
}

def scan_record(record_id: str, text: str) -> list:
    resp = requests.post(
        API_URL,
        json={
            "api_key": "YOUR_API_KEY",
            "api_type": "pii_detection",
            "text": text,
            "entities": list(EXEMPTION_MAP.keys()),
            "mask_mode": "replace",
            "threshold": 0.4,
            "custom_instruction": "Do not flag names of agency officials acting in their official capacity.",
        },
        timeout=30,
    )
    data = resp.json()
    # Produce analyst worksheet rows: candidate span + suggested exemption
    return [
        {
            "record": record_id,
            "span": (e["start"], e["end"]),
            "type": e["type"],
            "suggested_exemption": EXEMPTION_MAP[e["type"]],
            "confidence": e["confidence"],
        }
        for e in data["detected_entities"]
    ]

for row in scan_record("2026-FOIA-04412-p0038", open("p0038.txt").read()):
    print(row)

JavaScript — Open-Data Pre-Publication Gate

// Block dataset publication while any free-text field still contains PII
async function gateDatasetRow(freeTextField) {
  const resp = await fetch("https://piidetectionapi.com/api/moderate.php", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      api_key: process.env.PII_API_KEY,
      api_type: "pii_detection",
      text: freeTextField,
      exclude_entities: ["URL"],  // portal links are expected in this column
      mask_mode: "redact",
      threshold: 0.4
    })
  });

  const data = await resp.json();
  return {
    publishable: data.entities_detected === 0,
    cleaned: data.anonymized_text,
    flagged: data.detected_entities.map(e => e.type)
  };
}

gateDatasetRow(
  "Inspection note: spoke with owner Luis Herrera (cell 555-410-2276) at 2203 Palm Ct."
).then(console.log);  // publishable: false — route to review

Best Practices for Public-Sector Deployments

Keep humans on the balancing test. Configure detection as a flagging layer: every hit becomes a proposed redaction with a suggested exemption code, and a trained analyst approves, modifies, or releases. This preserves the legal judgment FOIA requires while eliminating the mechanical search that consumes most review time.

Encode agency context in custom instructions. Officials acting in their official capacity generally lack a protectable privacy interest in that capacity. A custom_instruction such as "do not flag names of agency officials in their official capacity" keeps the worksheet focused on third parties — the names that actually require Exemption 6/7(C) analysis.

Tune thresholds by release channel. Documents leaving under FOIA get a recall-first threshold of 0.4 with analyst review absorbing false positives. Open-data pipelines get a hard zero-entity gate. Internal routing and triage can run at the default 0.5 where a missed phone number is recoverable.

Preserve the audit trail. Store each record's detected_entities array with the processing note. When a requester challenges a redaction — or a court asks how segregability was assessed — the agency can show a uniform, documented methodology rather than reconstructing individual analyst memory.

Deployment options for government data

Agencies that cannot route record text through a commercial cloud can run the same detection engine on-premise or within their own authorized cloud enclave, keeping all processing inside the agency boundary. The cloud API itself runs on GDPR-native certified infrastructure and never trains on request content. Reach out via the contact page to discuss deployment and procurement.

Government PII Detection FAQ

Common questions from FOIA officers, records managers, and agency technologists

Does the API decide which FOIA exemption applies?

No — and it shouldn't. The API finds and classifies personal identifiers; your workflow maps entity types to candidate exemption codes (as in the Python example above), and a trained analyst makes the withholding decision, including the foreseeable-harm and public-interest analysis the statute requires. That separation is deliberate: it keeps legal judgment with accountable humans while automating the part of review that is pure pattern recognition at scale.

How is Exemption 6 handled differently from Exemption 7(C)?

The detection layer is identical — both exemptions protect the same kinds of identifiers. The difference lives in your routing rules: records flagged as law-enforcement-originated can map hits to 7(C), whose "could reasonably be expected to constitute" standard tips more readily toward withholding, while general records map to Exemption 6's stricter balance. Teams typically maintain two exemption maps keyed off the record's source system and apply the appropriate one per batch.

Can this handle a 100,000-page responsive set?

Yes. Each request accepts up to 50,000 characters, and requests parallelize freely, so a hundred-thousand-page set is an overnight batch job rather than a quarter-long review project. Typical per-page latency is a few hundred milliseconds; throughput scales with the concurrency your plan allows. See pricing for volume tiers, and use the live demo to estimate hit density on a sample of your records.

What about scanned and typewritten legacy records?

Run OCR first, then send the extracted text. Because detection is context-aware rather than regex-bound, it tolerates the character-level noise typical of typewriter-era scans — an SSN with an O for a zero still sits in an SSN-shaped context and gets flagged. For heavily degraded microfilm-sourced text, lower the threshold and sample-audit pages with unusually low hit density. Our document scanning guide covers the full OCR pipeline.

Does detection work for correspondence in languages other than English?

Yes — more than 60 languages are supported, which matters for agencies serving multilingual constituencies and for consular, immigration, and trade records. Mixed-language documents are handled in a single request with no language parameter needed. The supported languages page lists current coverage.

How do we defend automated detection in FOIA litigation?

The same way agencies defend search adequacy: documented methodology plus human review. Persist the per-record entity output (types, offsets, confidence scores), the threshold and instruction configuration used, and the analyst disposition of each hit. That record demonstrates a uniform, reasonable process for locating privacy material and assessing segregability — a stronger evidentiary posture than testimony about what individual reviewers remember reading. Guidance on validating detection accuracy itself is in our precision and recall guide.