Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

CI/CD pipelines: integrate, build once, promote

Merge small changes often and build and test every one, stop the line when main is red, build the artifact once and promote that same digest through every environment, keep the pipeline as code with gates between stages, and choose between continuous delivery and continuous deployment.

Two green branches, one red main

Two changes can each pass every test and still break when they meet. On an appointment-booking product for physiotherapy clinics, engineers worked on branches for a week or two and merged them on Thursdays. Thursday afternoon went on merge conflicts, Friday on tests that had passed on every branch and failed on the combination: one branch had renamed a field that another had started to read. A bug found on Friday could have come from any of 140 commits.

The changes had been tested alone and met each other too late. Continuous integration (CI) moves that meeting forward. Everyone merges small changes into the main branch at least daily, and every push and every pull request is built and tested automatically by a pipeline, a defined sequence of steps that a CI server runs on its own runners, not by a developer who remembers to. What it tests is the combination: a pull request as merged with the current main, where the CI supports it, and main itself again after every merge. A conflict between two changes now shows up within hours, between two small diffs, where it is cheap to find.

Small daily merges raise the obvious question about a feature that takes three weeks. It merges in pieces, switched off until it is ready; Feature flag services: targeting, rollouts & kill switches covers how a flag separates releasing a feature from deploying its code.

A red main stops the line

CI works only while main stays green. When a merge turns it red, every later change is tested on top of a broken base, and a second failure hides behind the first. So the team agreed that a red main stops the line: nobody merges until it is fixed, and the quickest fix is usually to revert the change that broke it and land it again with the fix. A rule like this survives only if the pipeline is fast, so the team runs the quick unit tests first, fails at the first error, runs the slower suites in parallel, and set itself ten minutes from push to result.

The rule's enemy is the flaky test, one that fails now and then on unchanged code, because of timing, test order or shared state. After a few weeks of pressing retry, people stop reading red runs, and the day a real failure looks like the flaky one, it gets merged. A flaky test is a defect: take it out of the blocking suite with a named owner and a deadline, fix it, put it back. Retrying it automatically until it passes keeps the pipeline green and the signal useless.

The release staging never saw

With CI in place, the team shipped through three environments: tests on every push, a staging deploy for the product manager to try, then production. Each stage checked out the commit and built it. One spring a release that had passed staging all week crashed in production at start-up. Same commit, different bytes: the production build ran four days after the staging one, the build did not use the lockfile, and a dependency declared as ^4.2.0 resolved to 4.3.0, published in between. Staging had tested one artifact and production ran another.

The rule that prevents it is build once, promote the artifact. The pipeline builds the image a single time, when the change merges to main, pushes it to the registry, and from then on refers to it by its digest, sha256:9f2c…, a hash of its content, rather than by a tag such as v2.14 or latest, which can be moved to other bytes. Staging deploys that digest; production deploys the same digest once staging has passed. What differs between environments is configuration, the database address or the payment provider's keys, supplied at deploy time and not baked into the image. Bake it in and every environment needs its own build, and the rule is gone.

Built once, pulled twice

The runner builds and tests one image; staging and production pull the same digest from the registry. The pipeline drives the production rollout, and a rollback deploys the previous digest.

Between stages sit gates, conditions a change must meet to move on: unit tests before the merge, integration tests and a scan of the image before staging, a smoke test and for some teams a person's approval before production. Each gate after the build records the digest it judged, and the next stage accepts only that digest, so nothing is promoted on a test that ran against other bytes. Rolling back is then the same move as deploying, which the pipeline can make without a manual scramble: point production at the previous digest, still in the registry as long as its retention rules keep it.

The pipeline is a file in the repository

The stages, their order and their conditions live in the repository, in a .github/workflows/ file, a .gitlab-ci.yml or a Jenkinsfile, rather than in settings clicked together in a CI server's web console. Pipeline as code gives the pipeline what the application already has: review, history and a way back. A change that drops the integration tests arrives as a diff that, where main requires review, someone has to approve, and a branch can try a new stage without touching anyone else's. The environments the pipeline deploys into can be declared the same way, which Infrastructure as code covers.

The pipeline holds credentials that push images and deploy, and a pull request can edit the pipeline file, so jobs that build untrusted pull requests run without those credentials and only the protected main branch can promote. Signing the digest the pipeline built and recording its provenance are Software supply chain: signing, provenance & scanning.

Delivery or deployment

The CD in CI/CD stands for one of two things, and they differ by one gate. In continuous delivery, every change that passes the pipeline is ready for production, and a person decides when it goes, with a button or a release schedule. In continuous deployment that step is gone: every change that passes goes to production on its own. The clinic team started with delivery and moved to deployment once rollback no longer needed anyone awake.

Deploying straight to 100% of traffic with no canary means a bad release hits everyone at once; blue-green at least makes the rollback a single traffic flip. Those rollout shapes and the database changes that keep rollback possible are Progressive delivery: blue-green, canary & rollback, and a canary is only as safe as the error rate and latency someone watches (Observability: metrics, logs & alerting).

Practise it as a flowchart

CI/CD Pipeline Flow is a flowchart Challenge: start on a code push, build, then a test decision where a failure stops and reports, parallel deploy steps, two production regions for instance, that merge before release, a rollback path for a failed deploy, and an End terminal for a successful release. Name the artifact in the build step, image sha256:…, and have every parallel deploy step use that one image. Code Deployment Platform puts the same pipeline into an architecture, with a scanner before the registry and Argo CD syncing the desired state from Git.

Practice this