L69 · Privacy and data classification

Module 10: People, Governance, and Emerging Threats · ~20 min · Lesson 69 of 71

🎯 What you'll learn

• What PII, NPI, PHI and PCI data are, and why one field can be all four
• How to label data (public, internal, confidential, restricted) so tools and people can act on it
• When to mask, tokenize, encrypt, pseudonymize or anonymize, and what each one really protects
• Why a prod copy in QA is a classic leak, and what to use instead

🔥 Why this matters

Testers want "realistic data", so a nightly Airflow job restores the prod policyholders snapshot into QA, where access is wide and debug logging is on. A year later a QA log bucket turns out to be shared too widely, holding 400,000 real SSNs. Nothing in prod was attacked; the data walked out. Most privacy failures look like this: data in a place nobody classified.

1. Four names for "sensitive": PII, NPI, PHI, PCI

Look at one document from a policyholders collection:

 {
   _id: "PH-100482",                     internal ID
   name: "Dana Testperson",              PII
   ssn: "000-12-3456",                   PII + NPI   (restricted)
   dob: "1984-03-09",                    PII
   zip: "02110",                         PII when combined
   policy: { no: "LP-778812",            NPI (shows a customer
             face_amount: 500000 },           relationship)
   underwriting: { smoker: false,        health data; PHI only
                   bmi: 24.1 },          under HIPAA (see below)
   card: { pan_token: "tok_9f2c...",     a token, not the PAN
           last4: "4242" }
 }

• PII (personally identifiable information) is anything that identifies a person, alone or combined with other data (NIST SP 800-122). Name, SSN, email, and also ZIP plus birth date.
• NPI (nonpublic personal information) is the GLBA term (L68): personal financial data a customer gave you or that comes from your relationship. Even "Dana is our customer" is NPI.
• PHI (protected health information) is HIPAA's term for identifiable health data held by a covered entity (health plans, providers, clearinghouses) or its business associates. Life insurers generally aren't covered entities, so their underwriting data usually isn't HIPAA PHI, though state insurance laws still protect it. At an insurer, the privacy team decides.
• PCI data is cardholder data (the PAN, primary account number, plus name and expiry) and sensitive authentication data (CVV, PIN), which must never be stored after authorization.

The labels overlap, each with its own rulebook, so you classify by handling level instead.

2. Classification levels and labels

Most companies use four levels. The names vary; the idea doesn't.

Two rules matter. The highest field wins: a collection with one restricted field is a restricted collection. And aggregation raises the level: a list of names is confidential, but names plus birth dates plus ZIPs for every customer is a breach.

Labels only get enforced if machines can read them. On AWS, tag resources DataClassification=restricted, allow only the four values with an Organizations tag policy, and use aws:ResourceTag conditions (ABAC, L27) to limit who can read restricted buckets.

MongoDB has no collection tags. Keep a data dictionary in git listing each field's level, with a CI check that fails when a new field has no entry. You can also put the level in a $jsonSchema validator's description, so it travels with the collection.

3. Collect less, keep less

The cheapest data to protect is data you don't have. Data minimization means collecting, returning and keeping only what a purpose needs:

• Don't collect it. Does the quote API need a full SSN, or would last four and DOB do?
• Don't return it. Response DTOs that list their fields (L41) stop "whole document" leaks.
• Don't log it. L60's redaction filter is the safety net, not the plan.
• Delete on schedule. A retention policy sets how long each record type is kept. Laws set minimums; privacy sets maximums. A legal hold pauses deletion during litigation.

Enforce retention in code: a MongoDB TTL index (expireAfterSeconds) on a date field, and S3 lifecycle rules for exports. Then remember the copies in backups, QA, logs and analytics (L53's crypto-shredding deep dive handles backups).

4. Masking vs tokenization vs encryption

All three hide a value. They differ in whether it can come back, and who can bring it back.

 SSN 000-12-3456
    │
    ├─ MASK ──────▶ ***-**-3456    one-way; for display
    │
    ├─ TOKENIZE ──▶ 731-88-2049    reversible only by asking
    │     (vault)    random token  the token vault (API + authz)
    │
    ├─ TOKENIZE ──▶ 518-40-7713    reversible by anyone with
    │   (vaultless,  same format   the FPE key
    │     FPE)
    │
    └─ ENCRYPT ───▶ AYc0x9…Qk==    reversible by anyone with
                    new format     the key (L15, L53)

Masking hides part or all of a value. Static masking rewrites a copy permanently (a QA dataset). Dynamic masking leaves storage alone and masks at read time, per caller: support sees ***-**-3456, fraud sees the full value. In MongoDB, build a view that projects the masked field and grant support find on the view only.

Tokenization swaps a value for a stand-in. In a vault-based design the token is random and the only link back is a lookup table in a hardened service, so detokenizing is an audited API call. A vaultless design computes the token with format-preserving encryption (FPE): a nine-digit SSN becomes another nine-digit number, so old schemas still validate, but the key is everything. NIST's FPE modes are FF1 and FF3-1 (SP 800-38G); a 2025 draft revision drops FF3-1 after new attacks, so prefer FF1. Systems that hold only card tokens and can't detokenize can leave PCI scope (L68).

Encryption (L15) is reversible by any key holder; L53's field-level encryption hides plaintext even from the database.

Choose by one question: who, if anyone, must ever get the real value back?

5. Pseudonymization vs anonymization

Analysts want to count claims per customer without seeing who the customers are.

Pseudonymization replaces identifiers with values someone could link back using a key or mapping. Anonymization means nobody can reasonably re-identify anyone. Under GDPR, pseudonymized data is still personal data; anonymous data is out of scope. (A 2025 EU court ruling nuanced this for recipients with no realistic way to re-identify; don't rely on it alone.) Two traps:

• Hashing isn't anonymizing. There are under a billion possible SSNs, so sha256(ssn) reverses by brute force in minutes. Use a keyed HMAC with the key kept outside the analytics account. That's a good pseudonym, and still personal data.
• Removing names isn't anonymizing. Latanya Sweeney estimated that about 87% of Americans are unique by ZIP code, birth date and sex alone. Re-identification joins your "anonymous" rows to a public dataset (voter rolls, social media) on those columns.

6. Non-prod data: the classic leak

Dev and QA have looser access, more people, debug logs and vendor tools. A prod copy there turns each of those into a breach path.

 PROD account                          QA account
 ┌───────────────┐   1. nightly    ┌──────────────────┐
 │ policyholders │ ──restore────▶  │ policyholders    │
 │ restricted,   │    snapshot     │ real SSNs, read  │
 │ tight IAM     │                 │ by 60 people     │
 └───────────────┘                 └───────┬──────────┘
                                   2. debug│logs, test exports,
                                           ▼ a vendor test tool
                                   3. bucket shared too widely

Better options, best first:

1. Synthetic data: generated records with realistic shape and no real person behind them (Python's Faker library, or a generator that follows your business rules).
2. Masked subsets: static masking applied inside prod before data moves, consistently (the same customer gets the same fake SSN everywhere, so joins still work).
3. Guardrails: policies that stop prod snapshots being shared to non-prod accounts, and a scanner for real-looking SSNs in QA (deep dive 2).

🧪 Worked example: one record, four audiences

GET /v1/policyholders/PH-100482 returns the whole Mongo document from section 1. The data dictionary says ssn, card and underwriting are restricted and dob is confidential.

Before: same response for every caller.

{"id": "PH-100482", "name": "Dana Testperson",
 "ssn": "000-12-3456", "dob": "1984-03-09",
 "underwriting": {"smoker": false, "bmi": 24.1},
 "card": {"pan_token": "tok_9f2c71", "last4": "4242"}}

After: one DTO per audience, with masking applied in the service (not the client):

 consumer (scope)        ssn            dob          underwriting
 portal (self)           ***-**-3456    1984-03-09   omitted
 support (support:read)  ***-**-3456    **/**/1984   omitted
 underwriting (uw:read)  token only     1984-03-09   full
 analytics export        HMAC pseudonym birth year   bmi band

Nobody gets the raw SSN in a response. A job that truly needs it (tax reporting) calls the token vault, which checks its role and logs the detokenization. A gateway such as Kong can't do this: only the service knows field levels and who is asking.

🛠 Try it (optional, ~5 min)

See why hashing isn't anonymizing. Save as rehash.py and run python3 rehash.py on your machine (standard library only, no network, fake SSNs):

import hashlib, hmac, secrets

leaked = hashlib.sha256(b"000-12-3456").hexdigest()
for n in range(10_000):                     # 000-12-0000..9999
    guess = f"000-12-{n:04d}".encode()
    if hashlib.sha256(guess).hexdigest() == leaked:
        print("plain hash reversed:", guess.decode())

key = secrets.token_bytes(32)               # keep this in a vault
tag = hmac.new(key, b"000-12-3456", "sha256").hexdigest()
print("HMAC pseudonym:", tag[:16], "... no key, no reversal")

The loop finds the SSN in milliseconds; all 10⁹ SSNs take minutes on a laptop. Without the HMAC key the search is hopeless, but the key holder can still link records: pseudonymous, not anonymous.

🧠 Key takeaways

• PII, NPI, PHI and PCI overlap; classify by handling level, and the highest field sets the level.
• Make labels machine-readable: AWS tags with a tag policy, a data dictionary for Mongo, CI checks.
• Minimize first: don't collect, return, log or keep what the purpose doesn't need.
• Masking can't be reversed, tokens can be reversed only through the vault or key, and encryption can be reversed by any key holder. Pick by who must get the real value back.
• Hashed or HMAC'd identifiers are pseudonymous, not anonymous; prod data doesn't belong in QA.

📖 Words to know

⏭ Next time

L70: what changes when your API has an LLM in front of it, and why a PDF can now give orders.

The main lesson said removing names isn't anonymizing. This section shows how privacy engineers measure how anonymous a dataset is, using a claims extract an actuarial team wants to share with an outside consultant.

Quasi-identifiers and k-anonymity

A quasi-identifier is a column that isn't an identifier on its own but helps single someone out when combined: ZIP, birth date, sex, job title. A dataset is k-anonymous if every combination of quasi-identifier values appears in at least k rows, so each person hides in a crowd of k.

The raw extract (fake data):

 zip    dob         sex  claim
 02110  1984-03-09  F    back injury
 02111  1984-07-21  F    back injury
 02110  1979-11-02  M    fracture
 02113  1981-05-30  M    anxiety

Every row is unique, so k = 1: anyone who knows a neighbor's ZIP and birthday learns their claim. Generalize (coarser values) and suppress (drop outliers) until each group has at least k:

 zip    birth decade  sex  claim
 021**  1980s         F    back injury
 021**  1980s         F    back injury
 021**  1970s-80s     M    fracture
 021**  1970s-80s     M    anxiety

Now k = 2. Each person is indistinguishable from at least one other.

Where k-anonymity breaks

Look at the two women: both have "back injury". If you know your neighbor is in the dataset and is a woman born in the 1980s, you learn her diagnosis without knowing which row she is. That's a homogeneity attack. l-diversity adds a rule: each group must contain at least l different sensitive values. Even that can leak when the values are similar (two kinds of back injury), which led to a further refinement, t-closeness. Each patch fixes one attack and costs utility.

The deeper problem: k-anonymity protects against the quasi-identifiers you listed. An attacker with a column you didn't think of (claim date plus a public news story) can still link.

Differential privacy: limiting what any one person changes

Differential privacy (DP) takes a different approach. Instead of publishing rows, you publish answers to queries (counts, averages) with calibrated random noise added, so the output is almost the same whether or not any single person is in the data. A parameter epsilon (ε) sets the trade-off: smaller ε means more noise and stronger privacy, and every query spends some of a total privacy budget.

 true count of back-injury claims, ZIP 021**:   37
 published with DP noise (ε = 1), three runs:   36, 39, 37
 Dana in or out of the data changes the true count by 1,
 which is hidden inside the noise

The US Census Bureau used DP for 2020 census data products, which shows both sides: a strong, provable guarantee, and a public argument about accuracy for small areas. For an engineering team it means: when outsiders need statistics, give them aggregates with DP or at least minimum cell sizes (don't publish any count under, say, 10), not row-level extracts.

Worked example: what the consultant actually gets

The consultant asked for "all claims with ZIP, DOB and diagnosis". After review they get:

• claims counts by 3-digit ZIP, birth decade and diagnosis group, cells under 10 suppressed;
• no row-level data leaves the account;
• if they need rows, a k ≥ 10 generalized extract under a contract that forbids re-identification, logged in the data-sharing register and treated as confidential, not public.

🧠 Extra takeaways

• Quasi-identifiers (ZIP, DOB, sex) re-identify people when combined.
• k-anonymity hides each person among k; l-diversity stops a group from sharing one sensitive value.
• Differential privacy adds calibrated noise to answers, with ε as the privacy budget.
• Prefer sharing aggregates with minimum cell sizes over row-level "anonymized" extracts.

Classification works for the fields you designed. The leaks come from the ones you didn't: a free-text notes field where an agent pasted an SSN, a CSV export in an S3 bucket from 2021, a debug log with a full request body. Data discovery means scanning for sensitive data where it isn't supposed to be.

Where it hides

 designed places               surprise places
 ┌───────────────────┐         ┌──────────────────────────────┐
 │ ssn field (CSFLE) │         │ notes: "cust SSN 000-12-..." │
 │ card.pan_token    │         │ S3: exports/old_report.csv   │
 │ dob (masked view) │         │ Airflow XCom / task logs     │
 └───────────────────┘         │ Kong logs with full bodies   │
   classified, protected       │ QA copies, support tickets   │
                               └──────────────────────────────┘
                                 unclassified, unprotected

Free-text fields are the worst offender: users type whatever they like, and a schema can't stop it.

Amazon Macie for S3

Amazon Macie scans S3 objects for sensitive data using managed data identifiers (SSNs, card numbers, bank accounts, AWS keys and many more) plus custom data identifiers you write as regex with keywords nearby, such as a policy-number pattern. Its automated sensitive data discovery samples objects across your buckets daily and gives each bucket a sensitivity score; discovery jobs do deeper scans of chosen buckets. Findings go to EventBridge and Security Hub, so a "restricted data in a bucket tagged internal" finding can open a ticket. Macie covers S3 only, not databases.

Worked example: a sampling scanner for your local MongoDB

For Mongo you write the scanner yourself. Run this only against a local MongoDB with fake data (docker run -p 27017:27017 mongo and insert a few test documents first):

import re
from pymongo import MongoClient

SSN = re.compile(r"\b\d{3}-\d{2}-\d{4}\b")
PAN = re.compile(r"\b(?:\d[ -]?){13,19}\b")

def luhn_ok(s):
    d = [int(c) for c in re.sub(r"\D", "", s)][::-1]
    t = sum(d[0::2]) + sum(sum(divmod(2 * x, 10)) for x in d[1::2])
    return t % 10 == 0

def walk(doc, path=""):
    for k, v in doc.items():
        p = f"{path}.{k}" if path else k
        if isinstance(v, dict):
            yield from walk(v, p)
        elif isinstance(v, str):
            yield p, v

db = MongoClient("mongodb://localhost:27017")["demo"]
for coll in db.list_collection_names():
    for doc in db[coll].aggregate([{"$sample": {"size": 200}}]):
        for path, val in walk(doc):
            if SSN.search(val):
                print(f"{coll}.{path}: SSN-like")
            m = PAN.search(val)
            if m and luhn_ok(m.group()):
                print(f"{coll}.{path}: card-like (Luhn ok)")

Typical output on test data:

policyholders.ssn: SSN-like
claims.notes: SSN-like
payments.memo: card-like (Luhn ok)

The first line is expected (that field is classified and encrypted). The other two are the real findings: a free-text field and a memo that need masking on input, cleanup, and an entry in the data dictionary. Print the location, never the matched value, or your scanner's output becomes the next leak (L60).

What scanners miss

Regex finds formats, not meaning. It flags 123-45-6789 in a test fixture and misses a birth date written as "March 9th". Luhn checks cut card false positives a lot, but not to zero. Sampling can miss rare documents. And no scanner finds a quasi-identifier problem (deep dive 1). Treat discovery as a smoke detector: it tells you where to look, then a person decides.

🧠 Extra takeaways

• Leaks come from unplanned places: free text, old exports, logs, XComs and QA copies.
• Macie scans S3 with managed and custom identifiers; databases need your own sampling scanner.
• Report where sensitive data was found, never the value itself.
• Use scanner hits to fix the source (input masking, data dictionary), not just to delete rows.