The metrics a lead watches, and the DORA caveat
What it is
A small set of measurements a team lead uses to see the system rather than the people, plus the discipline about what happens when any of them becomes a target.
The four DORA metrics are the standard starting point:
DEPLOYMENT FREQUENCY how often you deploy to production
LEAD TIME FOR CHANGES commit to running in production
CHANGE FAILURE RATE % of deploys causing a degradation
FAILED DEPLOYMENT time to restore after a failed
RECOVERY TIME deployment
The first two measure throughput; the second two measure stability. DORA's central and most-cited finding is that these are not in tension: high performers are better at all four, because the practices that make deployment frequent (small batches, automated testing, trunk-based development) are the same practices that make failures rare and recovery fast.
Commonly confused with productivity metrics. DORA measures the delivery system, not individuals, and applying it per engineer destroys it immediately, which is the caveat below.
Also commonly confused with a complete picture. All four are delivery metrics. None of them say anything about whether the software is useful, whether the team is sustainable, or whether quality is improving, and a lead watching only these is watching a quarter of the system.
The problem it solves
Without measurement, a lead's view of the team is anecdotal and biased toward whatever happened most recently or most loudly.
"How are we doing?"
-> "Pretty well, I think" is not an answer that survives
contact with a skip-level, and it is not one you can act
on.
"Should we invest in the deploy pipeline?"
-> Unanswerable without knowing what deploys cost now.
"Is the team overloaded?"
-> Everyone says yes when asked. The question is whether
it is worse than last quarter and by how much.
And the specific failure it prevents: optimising the visible thing. A team with no metrics optimises what the loudest stakeholder mentions, which is usually feature delivery, and the costs land on quality, on-call load and retention, which nobody is measuring.
Mechanics
The DORA four, with the performance bands
ELITE HIGH MEDIUM LOW
--------------------------------------------------------------------
Deploy frequency on demand daily to weekly to monthly
(multiple weekly monthly or less
per day)
Lead time for changes < 1 hour 1 day to 1 week to > 1 month
1 week 1 month
Change failure rate 0-15% 16-30% 16-30% 16-30%
Failed deployment < 1 hour < 1 day 1 day to > 1 week
recovery 1 week
Two things worth knowing about these bands. The cluster boundaries have moved between reports, so quoting a specific year's numbers as fixed is a mistake. And change failure rate barely differentiates the middle bands, which tells you it is the least useful of the four for positioning yourself.
How to instrument them without a platform:
DEPLOY FREQUENCY count deployment events. Trivial, and the
definition matters: production only, and
is a config change a deploy? Decide and
be consistent.
LEAD TIME commit timestamp to deploy timestamp for
that commit. Requires linking commits to
deploys, which is the only non-trivial
part. Measure the MEDIAN and the p85,
not the mean, because the distribution
is heavily right-skewed by long-lived
branches.
CHANGE FAILURE RATE deploys followed by a rollback, a hotfix,
or an incident, over total deploys. The
definition is where teams differ, so
write it down.
RECOVERY TIME from the failed deploy to service
restored, which is usually rollback time
plus detection time. Detection is often
the larger half and is the more
actionable one.
Measuring detection time separately from recovery time is the improvement worth making, because "we recover in 20 minutes" hides that 15 of them were before anyone noticed, and detection is fixed by alerting rather than by pipeline work.
The DORA caveat: AI-era findings
DORA's recent reports have complicated the throughput-stability story specifically around AI tooling.
The reported pattern: AI adoption correlates with increased
throughput (deployment frequency, lead time) AND with
increased instability (change failure rate, recovery time).
That is a DIFFERENT shape from DORA's original finding, where
the two moved together.
The interpretation to hold, and it is worth being precise because this is easy to overstate: these are correlational findings from a survey, not a controlled result, and DORA's own framing is that AI amplifies existing organisational characteristics rather than uniformly improving or degrading delivery. A team with strong testing, small batches and fast rollback gets faster; a team without them gets faster at shipping defects.
The practical consequence for a lead is concrete: pair every throughput metric with a stability guardrail and report them together. A team that reports deployment frequency alone during an AI tooling rollout will celebrate a number that is getting worse underneath.
And the mechanism worth naming: the bottleneck moves. When generation gets faster, the constraint shifts from writing code to reviewing it, and review queue depth is the metric that shows it. See the AI-era review queue.
Beyond DORA: what the four do not cover
A lead watching only DORA is watching delivery and nothing else. The additions that matter:
FLOW AND LOAD
interrupt rate unplanned work as % of capacity.
Above ~20% the team cannot plan.
work in progress items in flight per person. Above
~2 and everything slows.
cycle time distribution not the mean: the p85 and the
shape. A long tail means blocked
work, and the tail is where the
fixable problems are.
QUALITY
escaped defect rate bugs found in production per
release
time in code review often the largest single component
of lead time and rarely measured
test suite duration because a slow suite is a cause of
large batches
OPERATIONAL LOAD
pages per on-call shift above ~2 per night is unsustainable
% of pages actionable below ~50% means alert fatigue,
which is a reliability problem
toil % manual repetitive work as a share
of time
PEOPLE (watched, never targeted)
attrition, and WHO
1:1 themes over time
who is doing the invisible work
Interrupt rate is the one I would add first if I could add only one. It is the metric that explains why a team that "should" have capacity does not, it is invisible in every delivery metric, and it is directly actionable through rotation and triage.
Goodhart's law, which is the whole caveat
When a measure becomes a target, it ceases to be a good measure.
Every metric here has an obvious way to game it, and naming the gaming mechanism is what demonstrates you understand the risk:
METRIC HOW IT GETS GAMED
-------------------------------------------------------------
deploy frequency split one change into six deploys
lead time start the clock later, or merge
trivial commits frequently
change failure rate stop calling things failures, or
classify a hotfix as a feature
recovery time declare "recovered" earlier
velocity / story points inflate estimates. This one is so
reliably gamed that it should never
be a target at all
lines of code write more code. Obviously
PR count smaller PRs, which is sometimes good
and is not what you meant
test coverage assert nothing, cover everything
The rules that make metrics survive contact with incentives:
1. WATCH TRENDS, NOT VALUES.
"Lead time went from 4 days to 6" is information. "Lead
time is 6 days" against another team's 2 is not, because
the work is different.
2. NEVER MEASURE INDIVIDUALS ON TEAM METRICS.
DORA measures a delivery system. Applied per engineer it
is immediately gamed and it destroys the collaboration
the metrics exist to reflect.
3. PAIR EVERY THROUGHPUT METRIC WITH A QUALITY GUARDRAIL.
Deploy frequency with change failure rate. Lead time with
escaped defects. This is the same asymmetry as guardrails
in an experiment.
4. USE THEM TO ASK QUESTIONS, NOT TO GRADE.
"Lead time doubled this quarter, what changed?" is the
correct use. The answer is often something reasonable, and
sometimes it is a problem.
5. DO NOT REPORT THEM UPWARD AS PERFORMANCE.
The moment they are used for evaluation, they are a
target, and rule 1 through 4 stop working.
Rule 5 is the one leads get wrong under pressure, because a director asking "how is the team doing" wants a number, and providing DORA metrics as the answer converts them into a target within a quarter.
A worked example: reading four metrics together
QUARTERLY VIEW
deploy frequency 2.1/day -> 4.8/day +129%
lead time (p85) 3.2 d -> 1.9 d -41%
change failure rate 11% -> 19% +73%
recovery time 22 min -> 31 min +41%
interrupt rate 14% -> 23% +64%
review queue p50 4 h -> 11 h +175%
THE NAIVE READING
"Throughput more than doubled." Report it, celebrate.
THE ACTUAL READING
Throughput doubled AND stability degraded, which is the
AI-era pattern rather than the classic DORA one. And the
review queue tripling is the mechanism: generation got
faster, review did not, so changes are queuing and then
being reviewed under time pressure.
Interrupt rate rising 64% is the consequence: more
production issues means more unplanned work, which further
squeezes review capacity. It is a loop.
THE ACTIONS
a. Report throughput and stability TOGETHER, always, and
say the pattern out loud rather than leading with the
good number.
b. Attack the review bottleneck: cap PR size, because
review effectiveness collapses past roughly 400 lines
and generated PRs are often large; require authors to
explain generated code as their own; raise test
requirements on generated code, because tests are the
check that scales when volume rises and human review
does not.
c. Watch the loop: if interrupt rate keeps rising, the
stability problem is compounding and the throughput
gain will reverse.
The generalisable move: read the four together and look for the mechanism connecting them. Deployment frequency up and change failure rate up is not two facts, it is one story, and the review queue is where the story is visible.
Production evidence
The DORA / Accelerate research (Forsgren, Humble and Kim, Accelerate, and the annual State of DevOps reports) established the four metrics and the finding that throughput and stability move together in high-performing organisations. The performance bands are published annually and have shifted between reports, which is why quoting a specific year matters.
DORA's recent reports on AI adoption report the throughput-up, stability-down pattern and frame AI as an amplifier of existing organisational characteristics rather than a uniform improvement. These are survey-based correlational findings and should be cited as such.
The SPACE framework (Forsgren et al., 2021) is the explicit response to DORA being used too narrowly: it adds satisfaction, performance, activity, communication and efficiency, and its central argument is that no single metric captures developer productivity and that any individual-level metric will be gamed.
Goodhart's law, and Strathern's formulation of it, is the theoretical basis for rule 5, and the software-specific version is well documented: lines of code, story points and PR counts have all been used as targets and all failed the same way.
Google's re:Work and Project Aristotle findings on team effectiveness are the counterweight to delivery metrics: the strongest predictor of team performance was psychological safety, which no DORA metric captures.
The debate
The case for DORA: it is the only widely-validated set, it measures the system rather than people, and the four together resist the most obvious gaming because improving one at the expense of another is visible. It gives a lead a defensible answer to "how are we doing".
The case against: four delivery metrics is a narrow view of a team, they say nothing about whether the software is useful or the team sustainable, and the moment they are reported upward they become targets. Many teams instrument them, report them, and change nothing.
The case for no metrics: a lead who talks to their team knows more than any dashboard, and instrumentation is effort that could go into the work. This is right for a team of four and wrong at fifteen.
My position: DORA plus interrupt rate and review queue depth, watched as trends, never reported upward as performance, and always paired throughput with stability.
Interrupt rate is the addition I would make first because it explains the thing every other metric leaves unexplained: why a team that should have capacity does not. It is invisible in delivery metrics, it is directly actionable through rotation and triage, and above roughly 20 percent the team cannot plan at all, which makes every planning conversation a fiction.
The discipline I would hold hardest is rule 5: do not report these upward as performance. It is the one leads break under pressure, because a director asking "how is the team doing" wants a number and DORA metrics are the number available. Providing them converts them into a target within a quarter, and then they measure gaming rather than delivery. What I would report upward instead is outcomes and specific risks, using the metrics privately to know which risks to raise.
On the AI caveat, the precision matters: these are correlational survey findings and the useful framing is amplification rather than causation. A team with strong testing, small batches and fast rollback gets faster; a team without them gets faster at shipping defects. The concrete consequence for a lead is to pair every throughput metric with a stability guardrail and report them together, because a team reporting deployment frequency alone during an AI rollout will celebrate a number that is getting worse underneath.
And the reading discipline: look for the mechanism connecting the metrics rather than treating them as four facts. Deployment frequency up with change failure rate up is one story, and review queue depth is usually where it becomes visible, because when generation gets faster the bottleneck moves to review.
Where I would push back on a request for productivity metrics: there is no individual productivity metric that survives being a target, and SPACE's central argument is exactly that. If a director wants to know who is performing, the answer is manager judgement supported by artifacts, not a dashboard, and offering a dashboard is how you end up with a team optimising PR count.
Follow-up Q&A
"What metrics do you watch as a lead?" The DORA four as the base: deployment frequency and lead time for throughput, change failure rate and recovery time for stability. Plus two I would add immediately: interrupt rate, which is unplanned work as a share of capacity and is the metric that explains why a team that should have capacity does not, and review queue depth, which is often the largest single component of lead time and is rarely measured. And I watch trends rather than values, because comparing my lead time to another team's is meaningless when the work differs.
"What's DORA's central finding, and what's the caveat?" That throughput and stability are not in tension: high performers are better at all four, because the practices that make deployment frequent are the same ones that make failures rare. The caveat is that DORA's recent reports on AI adoption show a different shape, throughput up and stability down, and the useful framing is amplification: a team with strong testing and fast rollback gets faster, and a team without them gets faster at shipping defects. Those are correlational survey findings and I would cite them as such rather than as a causal result.
"So what do you do about it as a lead?" Pair every throughput metric with a stability guardrail and report them together, because a team reporting deployment frequency alone during an AI rollout will celebrate a number that is getting worse underneath. And watch the review queue specifically, because when generation gets faster the bottleneck moves from writing to reviewing, and queue depth is where that becomes visible before change failure rate shows it.
"How do you stop metrics being gamed?" Five rules. Watch trends rather than values. Never apply team metrics to individuals, because DORA measures a delivery system and per-engineer it is gamed immediately. Pair throughput with a quality guardrail. Use them to ask questions rather than to grade, so "lead time doubled, what changed" is the correct use. And do not report them upward as performance, which is the one leads break under pressure, because the moment they are used for evaluation the other four rules stop working.
"Your director asks how the team is doing and wants a number. What do you give them?" Outcomes and specific risks, not DORA metrics. Something like "we shipped the migration, the two remaining risks are the vendor dependency and the on-call load, and here's what I'm doing about each". I would use the metrics privately to know which risks to raise. Handing over the delivery metrics is how they become a target within a quarter, and after that they measure gaming rather than delivery.
"What would you add to DORA first?" Interrupt rate, unplanned work as a percentage of capacity. It explains what every other metric leaves unexplained, which is why a team that should have capacity does not, it is invisible in delivery metrics, and it is directly actionable through rotation and triage. Above about 20 percent the team cannot plan at all, which makes every planning conversation a fiction, and knowing that number changes what you commit to.
"How do you measure lead time properly?" Commit timestamp to production deploy for that commit, which requires linking commits to deploys and is the only non-trivial instrumentation. And measure the median and p85 rather than the mean, because the distribution is heavily right-skewed by long-lived branches and the mean tells you about the tail rather than the typical change. The p85 is usually the more actionable number because that is where the blocked work is.
"What about recovery time?" Split it, because "we recover in 20 minutes" often hides that 15 of them were before anyone noticed. Detection time and remediation time have completely different fixes: detection is alerting work and remediation is pipeline work, and they get conflated into one number that suggests the wrong investment. Detection is frequently the larger half and the more actionable one.
"Is there a productivity metric for individuals?" No, and I would say so directly. Every individual metric that has been tried, lines of code, PR count, story points, commits, has been gamed and has degraded exactly what it was meant to measure, and SPACE's central argument is that no single metric captures developer productivity. If a director wants to know who is performing, the answer is manager judgement supported by artifacts, and offering a dashboard instead is how you get a team optimising PR count.
Common misconceptions
"DORA measures productivity." It measures the delivery system. Applied to individuals it is gamed immediately and destroys the collaboration it exists to reflect.
"Throughput and stability trade off." DORA's central finding is that they do not, in high-performing organisations. The AI-era caveat is a different shape, not a return to the trade-off.
"The performance bands are fixed." They shift between annual reports, so quoting a specific year matters.
"Change failure rate is the key stability metric." It barely differentiates the middle bands, which makes it the least useful of the four for positioning.
"Report the metrics to leadership to show progress." That converts them into targets. Report outcomes and risks; use the metrics privately.
Interview delivery note
Give the four and the finding, then move quickly to what they miss, because that is where the judgement is: "The DORA four: deployment frequency and lead time for throughput, change failure rate and recovery time for stability. The central finding is that those aren't in tension, high performers are better at all four. But four delivery metrics is a narrow view of a team, and I'd add interrupt rate first, because it explains what all of them leave unexplained: why a team that should have capacity doesn't."
Name the AI caveat precisely, because overstating it is the common error: "And there's a recent caveat worth being careful about. DORA's reports on AI adoption show throughput up and stability down, which is a different shape from the original finding. Those are correlational survey results, and the useful framing is amplification: a team with strong testing and fast rollback gets faster, a team without them gets faster at shipping defects."
Give the practical consequence rather than the observation: "So as a lead I'd pair every throughput metric with a stability guardrail and report them together, because a team reporting deployment frequency alone during an AI rollout will celebrate a number that's getting worse underneath. And I'd watch review queue depth, because when generation speeds up the bottleneck moves to review, and that shows up before change failure rate does."
The rule that shows you have thought about incentives: "and I wouldn't report these upward as performance. That's the one leads break under pressure, because a director asking how the team is doing wants a number and these are the number available. The moment they're used for evaluation they're a target, and then they measure gaming. I'd report outcomes and specific risks, and use the metrics privately to know which risks to raise."
Further reading
- Forsgren, Humble and Kim, Accelerate, and the annual DORA State of DevOps reports, including the recent AI-adoption findings.
- Forsgren et al., "The SPACE of Developer Productivity" (2021), for why no single metric works and why individual measurement fails.
- Google re:Work and the Project Aristotle findings, as the counterweight to delivery metrics.
- Strathern's formulation of Goodhart's law, for the theoretical basis of the gaming rules.