L04 · Secure Design Principles

Module 1: Security Foundations · ~20 min · Lesson 4 of 71

🎯 What you'll learn

• Six design principles that show up in almost every security review: least privilege, defense in depth, fail secure, secure defaults, separation of duties, and zero trust
• What each one looks like in a stack of REST APIs, Kong, EKS, MongoDB, and AWS
• How to spot a fail-open bug in an authorization check, and how to fix it

🔥 Why this matters

In the Capital One breach (2019, covered in L47), the first bug was a misconfigured firewall. What turned a bug into the exposure of about 100 million people's data was a design choice: the firewall's IAM role could list and read far more S3 buckets than it ever needed. Most big breaches look like this. One thing breaks, and the design decides whether that's a bad day or a headline.

1. Least privilege

A batch job that only reads policy documents runs with an admin database account "because that's what worked in dev". Its credentials leak in a log file, and the attacker now owns every database.

Least privilege means every user, service, and job gets only the permissions it needs for its task, on only the resources it needs, for only as long as it needs them. It doesn't stop the credential leaking. It limits the blast radius: how much damage a compromised identity can do.

Here is the same job's AWS access, before and after. Both columns are valid IAM policies:

 ✗ OVER-PRIVILEGED           ✓ LEAST PRIVILEGE
 {                           {
  "Version": "2012-10-17",    "Version": "2012-10-17",
  "Statement": [{             "Statement": [{
   "Effect": "Allow",          "Effect": "Allow",
   "Action": "s3:*",           "Action": "s3:GetObject",
   "Resource": "*"             "Resource":
  }]                            "arn:aws:s3:::example-docs/stmts/*"
 }                            }]
                             }
 any action, any bucket      read only, one bucket prefix

The same idea in MongoDB: create a user with the built-in read role on one database, instead of root or readWriteAnyDatabase.

// mongosh, connected as an admin
use policies
db.createUser({
  user: "statement-reader",
  pwd: passwordPrompt(),               // never hard-code it
  roles: [ { role: "read", db: "policies" } ]
})

It applies to people too: one-hour admin access through IAM Identity Center beats a standing admin role. (AWS IAM in depth is L46; MongoDB users and roles are in L53.)

2. Defense in depth

Say a service trusts a gateway such as Kong to check tokens. Then someone adds a new Kong route and forgets the auth plugin. If the service never checks anything itself, the data is wide open.

Defense in depth means layering independent controls so that one failure doesn't expose the asset. A useful picture is the Swiss cheese model: each layer has holes, but the holes rarely line up. The key word is independent: three checks that all trust the same header are one layer, not three.

 Client
   │  HTTPS request + access token
   ▼
 ┌─ Layer 1: Kong ──────────────────────────────────────────┐
 │  validates token, rate-limits      if bypassed or        │
 │                                    misconfigured ───┐    │
 └─────────────────────────────────────────────────────│────┘
   │  mTLS                                             │
   ▼                                                   ▼
 ┌─ Layer 2: policy-service ────────────────────────────────┐
 │  re-verifies token, checks "may agent-77 see this        │
 │  policy?", validates input         still blocks it ✔     │
 └──────────────────────────────────────────────────────────┘
   │  least-privilege DB user, TLS
   ▼
 ┌─ Layer 3: MongoDB ───────────────────────────────────────┐
 │  auth required, network policy, encryption at rest       │
 │  a stolen app credential can only READ one database      │
 └──────────────────────────────────────────────────────────┘
   + Layer 4 everywhere: audit logs and alerts to catch what slips

3. Fail secure

Every check can fail to run: the entitlement service times out, a config file is missing, a library throws. Fail secure (also fail closed) means that when a security check can't reach a decision, the answer is "deny". Fail open means it lets the request through.

Saltzer and Schroeder called this fail-safe defaults in their 1975 paper The Protection of Information in Computer Systems, the origin of least privilege and several other principles here. Their rule: base access on explicit permission, not on the absence of a refusal. Careful with the words: in physical security a "fail-safe" door unlocks in a power cut so people can escape, while a "fail-secure" door stays locked. The worked example below shows the software version.

4. Secure defaults

Most systems run with whatever settings they shipped with, because nobody changes them. Secure defaults means the out-of-the-box configuration is the safe one, and being less safe takes a deliberate, visible change.

Apply the same rule to what you build: TLS on, debug off, and "deny" when a permission is missing.

5. Separation of duties

A developer writes a change that redirects a small fee to their own account, approves their own pull request, and deploys it. Every individual step was something they were allowed to do.

Separation of duties means no single person can complete a sensitive action alone: the person who makes a change is not the person who approves it, and ideally not the one who deploys it. Finance calls it maker-checker or the four-eyes principle, and auditors (SOX, L68) check it.

 developer ──writes──▶ pull request ──reviews──▶ second engineer
                                                       │
                         approval required, author ◀───┘
                         can't approve own PR
                                  │ merge to main
                                  ▼
                          CI/CD pipeline ──deploys──▶ prod
                     (no human holds prod deploy keys)

In GitHub this is branch protection: required approvals, "dismiss stale approvals" or "require approval of the most recent push" so nobody slips in a commit after review, and no admin bypass. (Pipeline security is L57.)

6. An introduction to zero trust

The old model was a castle and moat: anything inside the corporate network or VPC is trusted. Once an attacker is inside (a phished laptop, a compromised pod), they can talk to everything.

Zero trust drops the idea that network location means trust. Every request, from a user or a service, must prove who it is and be authorized, every time. NIST describes it in SP 800-207, Zero Trust Architecture (2020). The short version: "never trust, always verify", least privilege per request, and assume the network is already breached. It's not a product you buy; it's the principles above applied everywhere. The first deep dive below shows what it means for service-to-service calls.

🧪 Worked example: an authorization check that fails open

A policy-service endpoint asks an entitlement service whether an agent is assigned to a policy:

// ✗ VULNERABLE: fails open
boolean allowed = true;                      // assumes "yes"
try {
    allowed = entitlements.isAssigned(agentId, policyId);
} catch (Exception e) {
    log.warn("entitlement check failed", e); // and carries on
}
if (!allowed) {
    throw new ForbiddenException();
}
return policyRepo.findById(policyId);

On a normal day this works. Then the entitlement service has a 10-minute outage, every call throws, allowed stays true, and every agent can read every policy, with nothing but warnings in the logs. An attacker who can cause the exception (say, with a malformed ID) gets that result on demand.

// ✓ FIXED: fails closed
boolean allowed = false;                     // deny unless proven
try {
    allowed = entitlements.isAssigned(agentId, policyId);
} catch (Exception e) {
    log.error("entitlement check failed; denying", e);
    throw new ServiceUnavailableException("authz unavailable");
}
if (!allowed) {
    throw new ForbiddenException();
}
return policyRepo.findById(policyId);

Three changes: the variable starts at false, the exception path stops the request, and the caller gets a 503 ("try again") rather than a 403 ("not allowed"), so the outage shows up on dashboards. Add a unit test that makes the client throw and asserts no policy comes back (L59).

🛠 Try it (optional, ~5 min)

Find one place in your own stack where each principle shows up, or is missing:

A "missing" finding is the useful kind. Keep it for the L05 threat model.

🧠 Key takeaways

• Least privilege doesn't prevent compromise; it shrinks the blast radius when it happens.
• Defense in depth needs independent layers; the gateway is one layer, not the whole defense.
• When a security check can't decide, deny. Watch for allowed = true before a try.
• Secure defaults matter because nobody changes defaults; know which defaults in the tools you use are unsafe.
• Separation of duties: the person who writes a change shouldn't be the only one who approves it.
• Zero trust: network location grants nothing; every request proves identity and gets authorized.

📖 Words to know

⏭ Next time

L05: threat modeling with STRIDE. You'll draw a data flow diagram of a REST API, mark its trust boundaries, and ask six questions that find design flaws before an attacker does.

The flat-cluster problem

In a typical EKS cluster, any pod can open a connection to any other pod's service. If policy-service accepts any request that arrives on its port, then a compromised marketing-banner pod in the same cluster can call GET /v1/policies/PL-100245 and get an answer. The network said "you're inside", so the service said "yes". Zero trust replaces that implicit trust with three explicit questions on every call: who is calling, how do I know, and are they allowed to do this?

Piece 1: workload identity

A workload identity is a cryptographic identity for a piece of software rather than a person. In Kubernetes it usually derives from the pod's service account. Two forms you'll meet:

• SPIFFE IDs, used by service meshes such as Istio and Linkerd, look like spiffe://cluster.local/ns/policies/sa/policy-service. The mesh issues each pod a short-lived X.509 certificate carrying that ID in its Subject Alternative Name field.
• IRSA or EKS Pod Identity maps a service account to an AWS IAM role, so a pod gets temporary AWS credentials for its role instead of sharing the node's role or a long-lived access key (L50).

Piece 2: mTLS proves the identity

In normal TLS only the server shows a certificate. In mutual TLS (mTLS) both sides do, so the callee learns cryptographically which workload is calling, and nobody in between can read or alter the traffic. Certificates live hours, not years, and rotate automatically. (Certificates and mTLS in detail are L11.)

Piece 3: per-request authorization

Identity alone isn't permission. The callee, or a sidecar acting for it, checks every request against a policy: this caller, this method, this path, and ideally this end user.

 claims-service pod                     policy-service pod
 ┌──────────────┐  1. mTLS handshake   ┌──────────────────────┐
 │ app          │     (both show certs)│ sidecar / app        │
 │ + cert:      │─────────────────────▶│ 2. who? SPIFFE ID =  │
 │ ns/claims/   │  GET /v1/policies/   │    ns/claims/sa/...  │
 │ sa/claims-svc│  PL-100245           │ 3. allowed? policy:  │
 └──────────────┘  + user token        │    claims-svc may GET│
                   (agent-77)          │    /v1/policies/*    │
                                       │ 4. object check:     │
                                       │    agent-77 assigned?│
                                       └──────────┬───────────┘
                                         ✔ 200 / ✗ 403

Step 3 is service-level authorization ("may this service call this API?"). Step 4 is still the object-level check from the main lesson ("may this user see this record?"). Zero trust needs both: a legitimate service acting for the wrong user is still a breach.

🧪 Worked example: an Istio authorization policy

In a mesh, step 3 can be declared rather than coded. This (illustrative) Istio AuthorizationPolicy lets only the claims service read policies, and only with GET:

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: policy-service-allow-claims
  namespace: policies
spec:
  selector:
    matchLabels:
      app: policy-service
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/claims/sa/claims-svc"]
    to:
    - operation:
        methods: ["GET"]
        paths: ["/v1/policies/*"]

Once any ALLOW policy selects a workload, Istio denies requests that match no rule, so the marketing-banner pod now gets a 403. Secure defaults, least privilege, and defense in depth all show up in those few lines. Without a mesh, the same ideas come from Kubernetes network policies (L51) plus OAuth client-credentials tokens with a narrow audience and scopes (L23).

🛠 Try it (~3 min, runs only on your machine)

Make a throwaway certificate carrying a SPIFFE-style ID, then read it back the way a mesh would:

cd "$(mktemp -d)"
openssl req -x509 -newkey rsa:2048 -nodes -days 1 \
  -keyout key.pem -out cert.pem -subj "/CN=policy-service" \
  -addext "subjectAltName=URI:spiffe://example.org/ns/pol/sa/pol-svc"
openssl x509 -in cert.pem -noout -text | grep -A1 "Alternative"
# → URI:spiffe://example.org/ns/pol/sa/pol-svc

What to notice: the identity is a URI in the certificate, not an IP address. The pod can move anywhere and keep its identity; an attacker with the pod's IP but not its private key can't claim it.

🧠 Extra takeaways

• Being inside the cluster or VPC should grant nothing on its own.
• Workload identity (SPIFFE IDs, IRSA, Pod Identity) gives software a verifiable name.
• mTLS proves that name on every connection; short-lived certificates limit theft.
• Authorize every request at two levels: may this service call this API, and may this user see this object?

"Always fail closed" is the right default, not a universal law

Fail closed protects confidentiality and integrity by sacrificing availability. Usually that's the right trade. But some controls fail open on purpose, and some fail-closed choices cause outages worse than the attack they prevent. The skill is choosing per control, on purpose, and writing the choice down.

Real defaults in common tools

Notice the pattern. Rate limiting mainly protects availability, so turning it off briefly is often safer than rejecting every customer. Authorization protects confidentiality and integrity, so it defaults to closed.

A decision path

 the security check can't reach a decision
               │
               ▼
 does it guard confidentiality or integrity     no
 of sensitive data or money?  ─────────────────────▶ FAIL OPEN
               │ yes                                 + alert,
               ▼                                     time-boxed
 can a bounded, lower-risk mode keep the        yes
 critical path running?  ──────────────────────────▶ FAIL SOFT
               │ no                                  caps, queue,
               ▼                                     + alert
 FAIL CLOSED (503 + alert)

Fail soft (graceful degradation) is the middle option most real systems use: keep working, but with reduced risk. A cached authorization decision with a 60-second lifetime, a lower transaction limit, or a "pending review" state are all fail-soft designs.

🧪 Worked example: the fraud-scoring service is down

A payments API calls a fraud-scoring service before releasing each payout. It times out. Options:

The fail-soft row is what a thoughtful team usually picks, and the important parts are the bound (a hard cap) and the alert (someone knows it's happening). A silent fail-open with no cap is the version that ends up in an incident report.

Two warnings. First, if an attacker can trigger the failure (by sending huge requests, or malformed input that crashes the checker), any fail-open path becomes their exploit. Second, failing closed in a way that takes down production makes the security control the outage: availability is part of the CIA triad too (L01).

🛠 Try it (~3 min)

Pick one dependency your service calls before allowing something (entitlements, token introspection, a rate-limit store, a fraud score). Answer: What does the code do on a timeout today? Is that written down anywhere? Would you choose open, closed, or soft, and what's the compensating control?

🧠 Extra takeaways

• Fail closed is the default for controls protecting confidentiality, integrity, and money.
• Fail open can be reasonable for controls that mainly protect availability, like rate limits.
• Fail soft (bounded, degraded service plus an alert) is often the best real-world answer.
• If an attacker can trigger the failure, a fail-open path is an attack path.
• Know the failure defaults of your tools: Kong, Envoy, and Kubernetes webhooks differ.