Build once, promote by digest: a delivery pipeline you can roll back
A reference delivery path where the artifact tested in staging is exactly the one in production, and a rollback is a redeploy of the previous digest.
The examples are generic and illustrative. They are not taken from any client environment: adapt thresholds and settings to your own system.
The problem with rebuilding per environment
Pipelines that rebuild the application for each environment test one artifact and ship another. Dependencies resolve differently, a base image moves, a build flag differs — and the version in production was never tested anywhere. Build once, give the artifact an immutable identity, and promote that identity.
Reference path
A generic path from commit to production. Each stage either passes the same artifact on or stops it.
- 01CommitPull request with review and required checks.
- 02BuildOne build per commit; the image is pushed and its digest recorded.
- 03VerifyTests, dependency and image scans, signature.
- 04StagingThe digest is deployed; smoke and integration tests run.
- 05ProductionThe same digest, rolled out progressively.
- 06ObserveError budget and alerts decide: continue or roll back.
Digests, not tags
Tags such as latest or v2 are mutable labels; a digest (sha256:…) is the hash of the image content. Deployment manifests should reference the digest, and the pipeline promotes an environment by changing that one value — ideally as a commit to the deployment repository, so every release is a reviewed, revertible change.
# build once and record the digest
docker buildx build --push --metadata-file meta.json \
-t registry.example.com/api:"$GIT_SHA" .
DIGEST=$(jq -r '."containerimage.digest"' meta.json)
# promote: every environment receives the same digest
kustomize edit set image api=registry.example.com/api@"$DIGEST"Database changes: expand, then contract
Rolling back code is easy only if the schema still fits the previous version. Split breaking migrations into steps that are each compatible with the running code: add the new column or table, deploy code that writes to both, backfill, switch reads, and only in a later release remove the old structure.
Progressive delivery and rollback
- Roll out to a fraction of traffic or replicas first, and compare error rate and latency with the current version.
- Tie the decision to the SLO: a fast burn rate during the rollout stops the promotion automatically.
- A rollback is a redeploy of the previous digest — the same, already tested path — not a hotfix written under pressure.
- Keep configuration and secrets out of the image, so the same artifact runs in every environment.