Your DORA Metrics Can't See Your AI Investment

Here's a puzzle from the front lines of AI adoption. Faros AI analyzed over 10,000 developers across more than a thousand engineering teams and found that developers using coding agents completed 21% more tasks and merged nearly twice as many PRs. And yet, across those same teams, DORA metrics barely moved.

More output, same delivery performance. If that sounds impossible, welcome to the measurement crisis that's about to eat your next board slide.

The metrics were built for a different question

DORA metrics — deployment frequency, lead time, change failure rate, time to restore — answer the question "how healthy is our delivery system?" They were never designed to answer "is this specific investment paying off?" For a decade that distinction didn't matter much, because most interventions (CI improvements, trunk-based development, better on-call) affected the whole system and showed up in the whole-system numbers.

AI assistance breaks that assumption in two ways.

First, attribution. Your DORA dashboard has no idea whether a given commit was AI-assisted or hand-written. When lead time improves 15%, you cannot say whether that's the coding agents, the new hire, or the fact that the gnarly migration finally shipped. When it doesn't improve, you can't say whether AI is failing or whether it's working brilliantly on tasks that were never on the critical path.

Second, displacement. AI speeds up code production, but code production usually isn't the bottleneck. If your constraint is code review, staging environments, or a release train that leaves twice a week, agents merging 98% more PRs just builds a bigger pile in front of the same gate. Output went up; throughput didn't. Your DORA metrics are correctly telling you the system didn't get faster — while completely hiding the fact that one stage of it got dramatically faster and your bottleneck moved.

This is exactly what Google's DORA team found in their 2026 ROI report, drawn from nearly 5,000 practitioners: the returns from AI come from the surrounding organizational system — platform quality, workflow clarity, team alignment — not from the tools alone. Strong foundations amplify AI; weak ones absorb it.

What I'd instrument before (or at least during) rollout

The teams getting a clear ROI signal made one unglamorous decision early: they made attribution a prerequisite for adoption, not an afterthought. Concretely:

Tag AI-assisted work at the source. Commit trailers, PR labels, or tool-level telemetry that marks which changes had agent involvement and roughly how much. It doesn't need to be perfect — "agent-authored / agent-assisted / human-only" is enough resolution to be useful. Retrofitting this six months in is nearly impossible; the history is gone.

Track the bottleneck stage explicitly. Measure review pickup time and review cycle count separately from overall lead time. If agents double PR volume, this is where the pressure shows up first. A rising review queue with flat lead time is your early warning that you're buying speed at one stage and paying it back at another.

Pair every speed metric with a quality metric. Change failure rate and revert rate, segmented by the same AI-assistance tags. If agent-assisted PRs merge 40% faster but revert 3x more often, that's not a win — and blended averages will hide it completely.

Measure where the freed time goes. This is the one nobody does. The DORA team frames AI ROI as unlocking human capacity by offloading toil. Fine — but unlocked capacity is only worth something if it lands somewhere. Track what your teams do with it: more tech-debt burn-down? Faster incident follow-ups? More features started? If you can't see it, the CFO is entitled to assume the answer is "longer lunches."

The conversation this changes

The practical payoff is what happens in your next budget cycle. Leaders who walk in with "we spent $400K on coding agents and DORA is flat" lose the room. Leaders who walk in with "agent-assisted changes now make up 55% of merges, their lead time is 30% shorter, failure rate is at parity, and the review stage is our new constraint — here's the plan for it" get their renewal and their platform investment approved in the same meeting.

Same tools. Same spend. The difference is entirely in whether the measurement was designed before the rollout or improvised after someone asked.

Takeaway

Don't blame DORA — it's answering the question it was built for. But if you're spending real money on AI tooling, you need attribution (which work was AI-assisted), bottleneck visibility (where the extra output piles up), and quality segmentation (whether faster is also worse). Instrument those three things before you scale, and your metrics will finally be able to see the thing you're paying for.