Measuring CI: Cycle Time, Lead Time, and Failed Builds

You cannot improve a pipeline you cannot measure. The founding numbers of continuous delivery are lead time, cycle time, and change-failure rate, and they convert CI from a gut feeling into a set of trends you can move. These are the same metrics that power the DORA research on elite engineering teams, so they are worth taking seriously.

Milestones aging

Four terms separate the valuable parts of delivery:

• Lead time: from a change being committed to it being available to users.
• Cycle time: the part of that within the team's direct pipeline mechanics — build, test, review, deploy.
• Change-failure rate: the share of changes that cause a failure once shipped or require a rollback.
• Mean time to recovery: how long it takes to get back to health after a failure.

The distinction is useful because "our deploy takes a day" conflates, say, a four-minute build with an eighteen-hour queue. You need to know which spans to attack.

Instrumenting the timing

Good CI systems expose timing for each stage. Collect it: build time, test time, review time, deploy time. Then see where lead time actually goes. If tests are 80 percent of cycle time, optimizing the build is a waste. Getting the data is normally a query over your CI tool's API or a small logger in your pipeline.

import time, json

start = time.time()
# ... run the build step ...
payload = {
    "stage": "build",
    "seconds": round(time.time() - start, 2),
    "commit": os.environ.get("GIT_SHA", "unknown"),
    "branch": os.environ.get("GIT_BRANCH", "unknown"),
}
requests.post("https://metrics.internal/ingest", json=payload)

(Sometimes the pipeline API already emits these events; the logger above is for when it does not.) Feed the events to a small dashboard so trends are visible weekly, not reconstructed after a quarter.

Failed-build rate and the quarantine link

Track both the raw number of failed builds and their cause. A high rate of flaky-test failures points at your quarantine policy; a high rate of real integration failures points at your test coverage or your merge discipline. Break failure out by cause so you can act: infrastructure failures, flaky tests, and genuine regressions each warrant a different response. The earlier quarantine post is the mitigation for the flaky bucket; here you just want to count it.

Change-failure rate is the output metric

The pipeline metrics only matter if they move release outcomes. Change-failure rate — what fraction of releases need a fix or rollback — ties it together. If cycle time drops but change-failure rate rises, you have traded speed for stability at the wrong price. The healthy target is fast enough to be competitive and stable enough that releases are boring. Publish both toward the same board so nobody optimizes one at the other's expense.

Establishing a baseline before changing anything

Improvement starts with a baseline: track the four numbers for a couple of weeks before changing the pipeline. Otherwise you cannot tell whether a tweak helped or just shifted the noise. Once baseline is set, change one thing at a time and watch the effect. Fast, small experiments — speed up the build, quarantine one flake, trim review queue — are how pipeline engineering behaves like product engineering.

Takeaway: measure lead time, cycle time, failure rate, and recovery time, then change one thing at a time against a fixed baseline. Put the dashboard and the services it monitors on a Netbay VPS from Lucknow DC01, up in under 60 seconds at netbayhosts.in, and watch the trends move.