L15 · Symmetric Encryption: AES, Modes, and Nonces

Module 3: Cryptography Essentials · ~20 min · Lesson 15 of 71

🎯 What you'll learn

• What AES actually does (scramble one 16-byte block), and why that alone isn't enough
• How the block cipher modes ECB, CBC, CTR and GCM differ, and what each one leaks or protects
• The IV and nonce rules for each mode, and what goes wrong when you break them
• How to encrypt a MongoDB field correctly in Java or Python: AES-GCM, a fresh nonce, and AAD

🔥 Why this matters

A team needs to encrypt Social Security numbers in a customers collection. A developer writes Cipher.getInstance("AES"), loads a key from config, and the field turns into unreadable base64. Code review passes because "it's encrypted." Months later a security review finds two customers with the same ciphertext in ssn. Anyone who can read the database can tell which records share an SSN, and anyone who can write to it can copy an encrypted SSN into another document without the key. AES was fine; the way it was used was not. That's where most real encryption bugs live.

1. One key, both directions

Symmetric encryption uses the same secret key to encrypt and to decrypt. Whoever holds the key can do both; whoever doesn't can do neither.

It's fast (gigabytes per second with the AES instructions in modern CPUs), so it does the heavy lifting: TLS 1.3 records after the handshake (L10), EBS volumes and S3 objects at rest, MongoDB encrypted fields. The hard part, getting the same key to both sides secretly, belongs to L16 (asymmetric crypto) and L19 (key management). Today assumes the key is already in place.

2. AES is a block cipher: one block at a time

AES (Advanced Encryption Standard) is a block cipher: a function that takes a key and exactly one 16-byte block and returns another 16-byte block. NIST standardized it in 2001 (FIPS 197) after a public competition won by the Rijndael design. Keys are 128, 192 or 256 bits long.

             key K (16, 24 or 32 bytes)
                   │
                   ▼
  16 bytes  ┌──────────────┐  16 bytes
  plaintext │    AES_K     │  ciphertext
  ─────────▶│ 10/12/14     │──────────▶  looks random without K
  "Doe amou"│ rounds       │  9f 3a c1 ...
  "nt=$500."└──────────────┘
                   ▲
     decrypt runs it backwards, only with the same K

Think of the key as choosing one secret codebook out of an astronomically large library: each possible 16-byte input has exactly one entry. Technically, AES is a keyed permutation: for each key, it's a reversible one-to-one mapping on 2^128 possible blocks that's infeasible to tell apart from a random mapping without the key.

Notice what that definition leaves out. Real data isn't 16 bytes long, and the same input block under the same key always gives the same output block. A mode of operation is the recipe that uses the block cipher to encrypt a whole message, and the choice of mode matters more than AES-128 versus AES-256.

3. Four modes you'll meet

ECB: each block on its own

ECB (Electronic Codebook) splits the message into blocks and encrypts each one separately. Identical plaintext blocks become identical ciphertext blocks, so the pattern of the data shows through. The famous demonstration is the "ECB penguin": a bitmap of the Linux mascot encrypted with ECB, in which the penguin is still clearly visible.

  plaintext blocks         ECB ciphertext        CBC/GCM ciphertext
  ┌────────────────┐       ┌────────────────┐    ┌────────────────┐
  │ACCT=C-20417;OK!│ ───▶  │fd91bac126b7afe8│    │490f8add620bde30│
  │ACCT=C-20417;OK!│ ───▶  │fd91bac126b7afe8│    │1a73d16af26418e8│
  │ACCT=C-20488;OK!│ ───▶  │4cfd4675891a936d│    │fde44c31cfbd2f45│
  └────────────────┘       └────────────────┘    └────────────────┘
                           rows 1 and 2 match:   no visible pattern
                           pattern leaks

(Real openssl enc output, first 8 bytes of each block.) ECB is never right for data, yet some libraries make it the default, as the worked example shows.

CBC: chain the blocks together

CBC (Cipher Block Chaining) XORs each plaintext block with the previous ciphertext block before encrypting it. The first block has no predecessor, so it's XORed with an IV (initialization vector), a random 16-byte value sent along with the ciphertext.

   P1                P2                P3
   │                 │                 │
   ▼                 ▼                 ▼
  (⊕)◀── IV    ┌───▶(⊕)        ┌───▶(⊕)
   │           │     │         │     │
   ▼           │     ▼         │     ▼
 [AES_K]       │   [AES_K]     │   [AES_K]
   │           │     │         │     │
   ▼           │     ▼         │     ▼
   C1 ─────────┘     C2 ───────┘     C3
   each ciphertext block is mixed into the next plaintext block

Now equal plaintext blocks encrypt differently. CBC needs padding (filler bytes added so the message is a multiple of 16) and has no integrity check, which you'll break in the Try it.

CTR and GCM: turn AES into a stream

CTR (counter mode) encrypts a nonce plus a counter (1, 2, 3...) to produce a keystream, then XORs it with the message. No padding is needed, but it's malleable, as the L01 deep dive showed with the $500 to $900 bit-flip.

GCM (Galois/Counter Mode) is CTR encryption plus an authentication tag computed over the ciphertext with a fast keyed hash called GHASH. It's the AEAD from L01: any change to the ciphertext makes decryption fail.

4. IVs and nonces: the rules that bite

A nonce ("number used once") is a value that must never repeat under the same key. An IV is the broader term for a mode's starting value; each mode sets its own rule:

• CBC: the IV must be unpredictable, so generate it fresh from a secure random generator for every message. TLS 1.0 used the previous record's last ciphertext block as the next IV, which was predictable, and the 2011 BEAST attack exploited exactly that.
• CTR and GCM: the nonce must be unique per key. For GCM, use 12 bytes (96 bits). A random 12-byte nonce is fine, but NIST SP 800-38D caps random nonces at 2^32 encryptions per key so the chance of a collision stays negligible; after that, rotate the key.
• Neither is secret. Store the IV or nonce in plain sight next to the ciphertext. The receiver needs it to decrypt.

Reusing a GCM nonce is a disaster: an attacker learns the XOR of the two plaintexts and can forge tags. The first deep dive shows how.

5. Associated data: bind the ciphertext to its context

You met AEAD in L01. The "AD" part is worth a closer look. Associated data (AAD) is extra data that is authenticated by the tag but not encrypted, such as a record ID, a field name or a key version. Decryption succeeds only if the caller supplies exactly the same AAD.

That's how you stop the cut-and-paste attack from the opening. Encrypt each SSN with AAD customers/C-20417/ssn. If someone copies that ciphertext into customer C-20488's document, the application decrypts it with AAD customers/C-20488/ssn, the tag doesn't match, and decryption fails. Confidentiality wasn't at risk in that attack; integrity of where the data belongs was.

🧪 Worked example: encrypting a field in a policy service

Vulnerable (Java):

// "AES" alone = AES/ECB/PKCS5Padding in the default provider
Cipher c = Cipher.getInstance("AES");
c.init(Cipher.ENCRYPT_MODE, key);
byte[] ct = c.doFinal(ssn.getBytes(UTF_8));
doc.put("ssn", Base64.getEncoder().encodeToString(ct));

Three problems: ECB means equal SSNs give equal ciphertexts; there's no tag, so changes go unnoticed; and nothing ties the value to its record. A common "fix" makes it worse: switching to GCM with a private static final byte[] IV. Now every record reuses one nonce under one key.

Fixed (Java):

private static final SecureRandom RNG = new SecureRandom();

byte[] nonce = new byte[12];                  // 96-bit, fresh
RNG.nextBytes(nonce);
Cipher c = Cipher.getInstance("AES/GCM/NoPadding");
c.init(Cipher.ENCRYPT_MODE, key,
       new GCMParameterSpec(128, nonce));     // 128-bit tag
c.updateAAD(("customers/" + id + "/ssn")
       .getBytes(UTF_8));                     // bind to record
byte[] ct = c.doFinal(ssn.getBytes(UTF_8));   // ciphertext+tag
// store "v1:" + b64(nonce) + ":" + b64(ct); v1 = key version

On read, rebuild the same AAD from the document's _id. If the bytes were changed or moved, doFinal throws AEADBadTagException; treat that as a security event, not a retry.

The same in Python, run on this machine with the cryptography package (made-up values):

import os
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
g = AESGCM(key)                      # key: 32 bytes from KMS
nonce = os.urandom(12)
aad = b"customers/C-20417/ssn"
ct = g.encrypt(nonce, b"123-45-6789", aad)
# len(ct) == 27: 11 bytes ciphertext + 16 bytes tag

g.decrypt(nonce, ct, aad)            # b'123-45-6789'
g.decrypt(nonce, flipped, aad)       # InvalidTag
g.decrypt(nonce, ct, b"customers/C-20488/ssn")  # InvalidTag

In practice, before writing this yourself, check whether MongoDB's client-side field-level encryption (CSFLE) or Queryable Encryption fits: the driver then handles the algorithm, nonces and key lookups for you. Where the key itself lives is L19.

🛠 Try it (optional, ~5 min)

This runs only on your machine. On Linux, or on a Mac with Homebrew, openssl is OpenSSL (the sample below is OpenSSL 4.1); macOS's own /usr/bin/openssl is LibreSSL 3.3.6, which also accepts these commands.

mkdir -p /tmp/l15 && cd /tmp/l15
export DEMO_PASS='correct-horse-battery'  # demo only
printf 'policy=PL-100246 owner=C-20417 beneficiary=Jane Doe'\
' amount=$500.00\n' > claim.txt

# Encrypt: password -> key via PBKDF2, random salt and IV
openssl enc -aes-256-cbc -pbkdf2 -iter 600000 \
  -pass env:DEMO_PASS -in claim.txt -out claim.enc
xxd claim.enc | head -1   # "Salted__" + 8-byte salt
openssl enc -d -aes-256-cbc -pbkdf2 -iter 600000 \
  -pass env:DEMO_PASS -in claim.enc

-pbkdf2 turns the password into a key slowly, so guessing is expensive (L18). Without it, OpenSSL warns and uses a weak legacy derivation. -pass env: keeps the password out of your shell history. Now try GCM:

$ openssl enc -aes-256-gcm -pbkdf2 -pass env:DEMO_PASS -in claim.txt
enc: AEAD ciphers not supported
enc: Use -help for summary.

openssl enc has no way to store or check a tag, so it refuses AEAD modes. Now play attacker and change $500 to $900 without the password. The 5 sits in plaintext block 3, so flip the same position in ciphertext block 2 (file offset 16-byte header + 32 + 12 = 60):

python3 -c "d=bytearray(open('claim.enc','rb').read());\
 d[60]^=0x0C; open('evil.enc','wb').write(d)"
openssl enc -d -aes-256-cbc -pbkdf2 -iter 600000 \
  -pass env:DEMO_PASS -in evil.enc; echo "exit=$?"

policy=PL-100246 owner=C-20417 b▒▒627/▒▒▒▒▒▒▒▒a▒Doe amount=$900.00
exit=0

What to notice: exit code 0, no warning. Block 2 (eneficiary=Jane ) turned to garbage (different bytes each run), and the amount changed exactly as planned. Now flip the last byte instead (d[-1]^=0x01): you get bad decrypt, but only because the padding no longer looks right, and OpenSSL had already written the first 64 bytes of plaintext to the output. That's a padding check, not an integrity check: a lucky edit can still pass it.

🧠 Key takeaways

• AES encrypts one 16-byte block; the mode decides whether the whole message is actually safe.
• ECB leaks patterns, CBC and CTR hide them but can't detect tampering, GCM does both.
• CBC needs an unpredictable IV; GCM needs a 12-byte nonce that never repeats under a key.
• IVs and nonces aren't secret: store them with the ciphertext. Keys are secret.
• Use AEAD and put the record ID in the AAD so ciphertexts can't be moved between records.

📖 Words to know

⏭ Next time

L16: asymmetric cryptography, where one key locks and a different key unlocks, and how two strangers agree on a shared secret in public (explained with paint mixing).

Why a nonce matters so much in GCM

GCM's encryption half is CTR mode. The keystream depends only on the key and the nonce, not on the message. So two messages encrypted with the same key and nonce are XORed with the same keystream.

 C1 = P1 ⊕ KS(K, N)
 C2 = P2 ⊕ KS(K, N)        same K, same N  →  same keystream
 ─────────────────
 C1 ⊕ C2 = P1 ⊕ P2         the keystream cancels out

The attacker never needs the key. If they know or can guess one plaintext (a fixed JSON layout, a known field name, a test transaction they sent themselves), they get the other. This ran on this machine with Python's cryptography package, encrypting two messages under one key and one nonce:

c1 = enc(n, b"amount=$500.00;to=C-20417")
c2 = enc(n, b"amount=$750.00;to=C-20488")   # same n!
xor(xor(c1, c2), b"amount=$500.00;to=C-20417")
→ b'amount=$750.00;to=C-20488'

That's confidentiality gone for both messages.

The worse half: forged tags

GCM's tag is computed with a secret hash key H, derived from the encryption key (H = AES_K of an all-zero block). Each tag is a polynomial in H, masked with a value that depends on the nonce. With two different messages under one nonce, the mask cancels out, leaving an equation the attacker can solve for H. Antoine Joux described this in 2006, and it's sometimes called the "forbidden attack."

 1. capture two messages with the same nonce ──▶ (C1,T1) (C2,T2)
 2. subtract the tag equations              ──▶ mask cancels
 3. solve the polynomial for H              ──▶ hash key known
 4. compute valid tags for new ciphertexts  ──▶ forgeries accepted

Once H is known, the attacker can modify ciphertexts (using the bit-flipping from L01) and compute a tag that verifies. Integrity and authenticity are gone too, not just confidentiality.

This happened in the wild. In 2016, Böck, Zauner, Devlin, Somorovsky and Jovanovic scanned the internet and found 184 HTTPS servers repeating GCM nonces in TLS 1.2, including some at financial institutions, and demonstrated injecting content into their connections. They also found roughly 70,000 servers choosing nonces at random, safe only until enough data passes over one connection. TLS 1.3 fixed this by deriving each record's nonce from a sequence number defined in the protocol.

How nonces repeat in real systems

Nobody writes "reuse the nonce" on purpose. It happens like this:

The replica row is the sneaky one on Kubernetes. A counter-based nonce is perfectly safe in a single process, and broken the moment the deployment scales to two pods that share the key.

Defenses

• Use 12 random bytes from a CSPRNG (SecureRandom, os.urandom, crypto.randomBytes) for every encryption. That's the simplest safe default for app-level field encryption.
• Limit messages per key. Stay well under NIST's 2^32 cap for random nonces. Envelope encryption with a fresh data key per record or per batch (L19) makes this automatic.
• Use a misuse-resistant mode when you can't guarantee uniqueness. AES-GCM-SIV (RFC 8452) derives the IV from the nonce and the message, so a repeated nonce only reveals that two messages were identical. XChaCha20-Poly1305 uses a 24-byte nonce, large enough that random collisions are not a practical concern (it's widely implemented, for example in libsodium, but not yet an RFC).
• Never let the caller choose the nonce in a shared library. Generate it inside your encryptField() helper so nobody can pass a constant.

🛠 Try it (~5 min)

Search a codebase you own for GCMParameterSpec( or AESGCM( and check where the nonce argument comes from. If it's a constant, a field set once at startup, or a counter, it's worth a ticket.

🧠 Extra takeaways

• Same key + same nonce = same keystream, so C1 ⊕ C2 = P1 ⊕ P2.
• Nonce reuse in GCM also leaks the hash key H, which lets attackers forge tags.
• Counters break when replicas or restarts share a key; random 12-byte nonces avoid that.
• AES-GCM-SIV and XChaCha20-Poly1305 are safer when uniqueness is hard to guarantee.

The problem: you can't search random ciphertext

The main lesson's fixed code produces a different ciphertext every time, even for the same SSN. That's exactly what you want for confidentiality. But a support tool needs find({ ssn: "123-45-6789" }), and with random encryption the database can't match anything.

Deterministic encryption solves this: the same plaintext under the same key always produces the same ciphertext. The app encrypts the search value, and the database does an ordinary equality match on the ciphertext.

What MongoDB CSFLE offers

MongoDB's client-side field-level encryption gives you exactly this choice per field. The schema's algorithm value is one of:

• AEAD_AES_256_CBC_HMAC_SHA_512-Random: different output every time, no queries on the field
• AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic: same output for the same value, equality queries work

Both are authenticated (AES-CBC plus an HMAC-SHA-512 tag), so tampering is detected either way. The difference is only the IV: random in one, derived from the value in the other. MongoDB's docs also note that deterministic encryption can't encrypt whole objects or arrays.

"ssn": {
  "encrypt": {
    "bsonType": "string",
    "keyId": [UUID("0f6d...demo")],
    "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
  }
}

What it leaks: equality, and then frequency

Deterministic means anyone who can see the stored data learns which records share a value. For an SSN that's mostly harmless, since nearly every value is unique. For a field with few possible values, it's a leak. Imagine an attacker with a database dump of a policies collection where policyStatus is deterministically encrypted:

 ciphertext of policyStatus   count   attacker's guess
 ────────────────────────────────────────────────────────
 9c41...e0  ████████████████  81%  → ACTIVE   (most common)
 07ab...12  ██                11%  → LAPSED
 e3f9...7d  █                  6%  → PENDING
 51c0...aa                     2%  → IN_CLAIM (rarest)
 ────────────────────────────────────────────────────────
 encryption intact, but the values are recovered anyway

This is frequency analysis: matching ciphertext counts to known real-world distributions. MongoDB's documentation warns that low-cardinality fields encrypted deterministically are susceptible to it. The same goes for US state, gender, plan type or any yes/no flag. If two collections use the same data key, the attacker can also link records across them by matching ciphertexts.

Choosing per field

Other ways to search encrypted data

• Blind index: encrypt the field with random AEAD, and store a separate HMAC of the normalized value (with a different key) for lookups. It leaks equality like deterministic encryption, but you can truncate the HMAC so several values collide, trading false positives for less leakage. HMAC is L17.
• MongoDB Queryable Encryption: stores randomized ciphertext and keeps encrypted index data so the server can answer queries without seeing plaintext or identical ciphertexts. Current docs list equality and range queries, plus prefix, suffix and substring for strings; which ones are available and generally available depends on your server version, so check the release notes for yours. It still reveals some access patterns, such as which documents a query matched.
• Proxy fields: MongoDB's docs suggest storing something like last4ssn in plaintext for searching. Remember that it's unencrypted and reveals what it contains.

🛠 Try it (~5 min)

Pick one collection your team owns. For each sensitive field, write down: is it ever queried by exact value, and roughly how many distinct values does it have? Any field that is low-cardinality and would be encrypted deterministically is a frequency-analysis risk.

🧠 Extra takeaways

• Random encryption hides everything, including equality; deterministic encryption reveals equality so you can search.
• Deterministic encryption on low-cardinality fields is broken by frequency analysis.
• Use deterministic only for high-cardinality fields you must look up; use random for the rest.
• Blind indexes and Queryable Encryption are alternatives, each with its own, smaller leak.