Making the reliability investment case

"Make the case for spending 25 percent of your team's capacity on reliability to a product-focused VP."

What the question is testing

Not whether you value reliability. Everyone does, and saying so persuades nobody. Three things:

Can you speak the other person's language? A VP whose objectives are growth and roadmap does not have a slot for "technical debt". They have slots for revenue, cost, risk and speed. If your argument does not land in one of those, it does not land.
Do you have numbers or a feeling? "Things are fragile" is a feeling. "We spent 340 engineer-hours on incidents last quarter, which is 18 percent of capacity, and here are the three causes" is an argument.
Are you asking or negotiating? A request for 25 percent with no stated return, no duration and no exit condition sounds like a tax. A proposal with a number attached, a review date and a reversal condition sounds like an investment.

The failure mode is moral framing: quality matters, we should do it properly, we're accumulating debt. All true, all unpersuasive to someone whose quarter is measured in shipped features.

The core reframe

You are not asking for capacity. You are pointing out that you are already spending it, invisibly and at a worse exchange rate.

Unplanned work is capacity. Incident response is capacity. Rollbacks are capacity. The difference between planned reliability work and unplanned incident work is not whether you spend it, it is whether you chose to, whether it happens at 2pm or 2am, and whether it compounds.

That reframe is the whole argument and it should be the first thing you say, because it converts "give me 25 percent" into "here is where 18 percent is already going, and I can convert it into 8 percent".

The evidence to bring

Do this before the meeting. An hour of data collection is worth more than any amount of rhetoric.

1. The current unplanned-work rate. From the incident tracker and the sprint board: how many engineer-hours went to unplanned work last quarter, as a percentage of capacity. This is the single most powerful number and almost nobody brings it.

Q3: 340 engineer-hours on incidents and unplanned fixes
    = 18% of team capacity (8 engineers x 65 working days x 6h)
    Of which: 60% deploy-related, 25% capacity/scaling, 15% dependency failures

2. The trend. One quarter is an anecdote. Three quarters showing 11 percent, 14 percent, 18 percent is a trajectory, and a trajectory forecasts.

3. The delivery cost. Interruptions do not cost only their own hours. A paged engineer loses the rest of the day, and a team with unpredictable interrupts cannot commit reliably. If your on-time delivery rate has fallen alongside the incident rate, that correlation is the VP's problem, not yours.

4. The revenue or cost number, if you can get it. Ask finance or the product analytics team: what is an hour of degraded checkout worth? You often do not need precision. "Our checkout does roughly £180k an hour at peak, and we had 4.2 hours of degradation last quarter" is enough.

5. The comparison, carefully. DORA's research links change failure rate and recovery time to organisational performance, and the 2024 and later reports flag that AI-assisted development is raising throughput while degrading stability. Use it as context, not as proof: "the industry pattern is that throughput gains without a quality guardrail show up as change failure rate, which is what we are seeing" is defensible. "DORA says we need this" is not.

The pitch

Six sentences. Written down, rehearsed, under two minutes.

"We're spending 18 percent of the team's capacity on unplanned work right now, and it's been climbing for three quarters: 11, 14, 18. Sixty percent of it traces to deploy failures, which we don't catch because we have no automated rollback. I want to spend 25 percent for one quarter on three specific things: automated canary analysis with rollback, load-test gates in CI, and fixing the two dependency timeouts that caused four incidents. My expectation is that unplanned work drops below 8 percent, which gives us back roughly 10 percent of capacity permanently, and we'd review at the end of the quarter. If the number hasn't moved, I'd want us to stop and rethink rather than keep spending."

Every element is doing work:

Opens with a number, not a concern.
Shows a trend, so it forecasts rather than describes.
Attributes the cause, so it is not a vague plea for time.
Names three specific things, so it is a plan not a budget line.
States the expected return, so it is an investment with a hypothesis.
Bounds it to a quarter with a review, so it is reversible.
Offers to stop, which is the sentence that makes it credible.

That last one is counter-intuitive and it is the most important. A leader who says "and if it does not work we should stop" is a leader who will be believed the next time, because they have shown they are running an experiment rather than defending a position.

Translating into their language

You would say	Say instead
Technical debt	Interest we pay in delivery speed every sprint
The system is fragile	Change failure rate is 22 percent; one in five releases needs a fix
We need to refactor	This component causes 40 percent of incidents; here is the fix and its cost
Reliability work	Recovering the 18 percent of capacity we lose to unplanned work
Improve test coverage	Reduce the two-week average from bug report to fix
We're burning out on-call	We page 4.2 times per person per week; two engineers have asked to leave the rotation

The right-hand column is not spin. It is the same fact stated in the unit the listener manages.

Handling the pushback

"Can we do it after the launch?" Sometimes yes, and saying so builds credit. When no: "Two of the three items are launch prerequisites. The launch roughly triples traffic on the checkout path, and that path is where our capacity incidents already come from. I'd rather do the load-test gate before the launch than discover the limit during it." Tie the work to their goal rather than opposing it.

"25 percent is too much. Can you do it with 10?" Never refuse a smaller number outright; scope it and price it. "Yes, with 10 percent I'd do the canary and rollback work, which is the biggest single cause. That should take unplanned work from 18 to about 12. The load-test gate and the dependency timeouts would wait, and I'd expect the capacity incidents to continue at their current rate. If that's the trade you want, I'm fine with it, and I'd want to revisit after the launch." This is the strongest move in the conversation: you have accepted their constraint, priced it honestly, and put the consequence on the record without being obstructive.

"Why did this happen? Shouldn't it have been built properly?" Do not get defensive and do not blame predecessors. "Some of it was the right call at the time; we optimised for shipping and that was correct at the scale we had. What has changed is traffic and team size, and the practices that worked at 200 requests per second do not at 2,000. This is the cost of having been right earlier."

"How do I know this won't be a permanent 25 percent?" "Because I'm proposing a specific list, a number to hit, and a date to review. And separately I'd propose a standing split (say 60 percent features, 25 percent reliability and debt, 15 percent keep-the-lights-on) so this stops being a negotiation every quarter and becomes a budget we can both plan against." Offering the standing split is the move that turns a one-off ask into a durable operating agreement.

"The team should just be more careful." "Care doesn't scale and it isn't measurable. Sixty percent of our incidents are deploy failures we catch by hand after users notice. Automated canary analysis catches them in five minutes without anyone being careful, and it works at 3am." Systems over exhortation, stated plainly.

The strongest single tool: the error budget policy

If you can get this agreed, you never have this argument again.

An error budget policy is a pre-agreed rule: when the service is meeting its SLO, the team ships features at full speed; when the budget is exhausted, feature work pauses and the team works reliability until it recovers.

The property that makes it powerful is that it is negotiated in advance, in the calm, and then applied automatically. You are not asking for reliability capacity during an incident when the VP is annoyed; you agreed months earlier what would happen, and now you are both following a rule.

"I'd rather not have this conversation every quarter. Can we agree an error budget policy? We set an availability target together, say 99.9 percent, which is 43 minutes a month, and while we're inside it, the team ships at full speed and you never hear from me about reliability. When we blow through it, feature work pauses until we're back inside. That way the tradeoff is a rule we both signed rather than an argument we have when something is on fire."

The thing to say next, because it is the part that fails: the policy only works if leadership signs it before the budget runs out. Everyone agrees in principle and defects in practice, and the moment of truth is the first time the budget is exhausted during a launch quarter. Naming that up front, and asking for the commitment explicitly, is what separates a real policy from a slide.

What you must not do

Do not catastrophise. "We're going to have a major outage" is unfalsifiable and reads as manipulation. If you say it and no outage comes, you have spent your credibility; if it comes, you look like you knew and did not act.

Do not present it as a moral issue. "We should do this properly" invites the response "we should also ship", and neither side has an argument.

Do not hide the cost. If 25 percent means two features slip, say which two. Being the person who names the cost of their own proposal is worth more than the proposal.

Do not ask without a return. Any capacity request without a hypothesis about what improves is a tax, and taxes get cut.

A worked example: what happened next

A team ran this pitch and got 15 percent rather than 25, plus agreement to revisit.

They spent it on the single largest cause: automated canary analysis with rollback on the deploy pipeline. Six weeks of work for two engineers.

Result at the quarter review: unplanned work fell from 18 percent to 11 percent. Change failure rate fell from 22 percent to 9 percent. Mean time to recovery fell from 55 minutes to 8, because rollback became automatic rather than a human noticing.

The conversation at the review changed completely. They were no longer asking for capacity; they were reporting a return and proposing the next investment with a track record behind it. The VP approved the remaining items without much discussion, because the first tranche had produced a number.

The lesson worth generalising: take the smaller number, pick the highest-return item, deliver a measurable result, and come back. Insisting on the full 25 percent up front risks getting zero and having no evidence. Fifteen percent with a result beats 25 percent with an argument.

Production evidence

Google's SRE practice is the origin of the error budget policy, and the SRE book's framing is the one to borrow: the budget makes the reliability-versus-velocity tradeoff explicit and removes the need to argue it case by case. The workbook is blunt that the policy must be agreed by leadership in advance, and that a policy without that agreement is theatre.

DORA's research programme, published annually, links change failure rate and failed-deployment recovery time to organisational performance, which is the closest thing to industry-wide evidence that reliability and delivery speed are complements rather than opposites. Recent reports flag that AI-assisted development is raising throughput while degrading stability, which is directly usable context for this argument.

Nicole Forsgren, Jez Humble and Gene Kim's Accelerate is the book form of that research and the one a VP is most likely to have read, which makes it a useful shared reference rather than an appeal to authority.

The three-bucket budget (features, reliability and debt, keep-the-lights-on) with a published split is common practice across engineering organisations. The value is not the specific ratio, it is that publishing a split converts a recurring negotiation into a standing agreement.

The debate

The credible counter-argument, and you should be able to make it: a company that over-invests in reliability before finding product-market fit dies of irrelevance. For an early-stage product, an hour of downtime costs almost nothing and a missed market window costs everything. Reliability investment has a correct level and it is not "maximum".

The honest position is that the right level depends on what failure costs, and that varies by orders of magnitude across products and across the same product over time. A payments ledger and an internal dashboard should not have the same posture.

My position: make the argument in the unit the listener manages, price it, bound it, and offer to stop if it does not work. Take a smaller number and deliver a result rather than winning the full ask and having nothing to show. And push for an error budget policy, because the durable win is not this quarter's capacity, it is never having to make this argument again.

Making this case is the wrong move when the team's reliability problems are actually a scope or estimation problem in disguise, when the organisation is in a genuine survival moment where the roadmap is existential, or when you have made the same ask twice and not delivered on either. In the last case, deliver something small with your existing capacity first and come back with evidence.

Follow-up Q&A

"Make the case for 25 percent reliability investment to a product-focused VP." Open with the number, not the concern: we are already spending 18 percent on unplanned work and it has climbed for three quarters. Attribute it: 60 percent is deploy failures. Propose three specific items, state the expected return (unplanned work below 8 percent, so a net gain of about 10 percent capacity), bound it to a quarter with a review, and say explicitly that if the number does not move we should stop. The reframe underneath is that this is not new capacity, it is converting invisible expensive capacity into visible cheap capacity.

"They cut you to 10 percent. What do you do?" Accept it, scope it honestly, and put the consequence on the record without being obstructive: "with 10 percent I'd do the canary and rollback work, which should take unplanned work from 18 to about 12; the capacity incidents would continue and I'd want to revisit after the launch." Then deliver a measurable result, because 10 percent with a number beats 25 percent with an argument, and the second conversation is far easier when you have a track record.

"How do you avoid having this argument every quarter?" An error budget policy agreed in advance, plus a published three-bucket split. The policy turns the tradeoff into a rule both parties signed rather than a negotiation during an incident. The critical detail is that it must be agreed before the budget is exhausted, and the first time it binds during a launch quarter is when you find out whether it was real. Ask for that commitment explicitly rather than assuming it.

"What if the VP is right and the roadmap genuinely can't slip?" Then they are right, and saying so is worth more than the argument. Agree the roadmap, name which reliability risks we are accepting and what they would cost if they land, and put a date on revisiting. What I would push for is the smallest item with the highest return, usually automated rollback, because it converts long incidents into short ones for a few weeks of work, and I would ask for it on the grounds that it protects the launch rather than competing with it.

"How do you measure whether it worked?" The same numbers I opened with, so the comparison is like for like: unplanned work as a percentage of capacity, change failure rate, mean time to recovery, and pages per person per week. I would set the baseline before starting and report at the review whether or not it is flattering, because reporting a disappointing number honestly is what makes the next ask credible.

Common misconceptions

The biggest is that this is a persuasion problem. It is a measurement problem: the team that can state its unplanned-work percentage and its trend rarely has to persuade anyone, and the team that cannot will not win the argument however well they make it.

The second is that you should ask for what you need. Ask for what you can justify with evidence and deliver against, then come back. Credibility compounds; a large unfunded ask does not.

The third is that reliability and velocity are opposites. The DORA research is fairly consistent that high performers are better at both, because the practices that make deployment safe (automated testing, small changes, fast rollback) also make it fast. Framing it as a tradeoff concedes ground you do not need to concede.

Interview delivery note

Say this, and say the reframe first: "I wouldn't open by asking for capacity. I'd open by showing that we're already spending it: 18 percent of the team's time went to unplanned work last quarter, up from 11 two quarters ago, and 60 percent of it is deploy failures. Then I'd propose three specific fixes, state the return I expect (unplanned work under 8 percent, so a net gain of about 10 percent), bound it to one quarter with a review, and say that if the number hasn't moved we should stop rather than keep spending."

Then the two moves that make it a lead-level answer. Offering to stop, because it converts a request into an experiment and makes you credible. And the error budget policy: "and separately I'd push for an error budget policy, so this stops being a quarterly negotiation. The catch is that leadership has to sign it before the budget runs out, not during the incident, and I'd ask for that explicitly."

The depth signal is accepting the smaller number gracefully and pricing the consequence. Candidates who argue harder for the full ask sound like advocates; candidates who say "yes, and here is what we would not get" sound like people who have run a team.

The Staff & Team Lead Interview Reference