The real questions now sit on every CTO’s desk: are we actually gaining productivity? And is the ROI positive, given the money we’re pouring into tokens?

DX’s Q2 report is the most honest attempt I’ve seen at answering both. And the answers should make a lot of engineering leaders uncomfortable.

First, the data behind it

This isn’t a vibes-based survey. The report measures 500+ engineering organizations on the DX platform, from fewer than 50 to over 10,000 employees, across Europe, Asia and North America, and across industries including Finance, Retail and Healthcare. The reporting period covers April to June 2026, with trend data going back to Q3 2025.

It combines two complementary sources: system metrics pulled automatically from source control, CI/CD pipelines and AI tools, and survey metrics capturing what developers actually experience. Findings are organized around the DX Core 4 (Speed, Effectiveness, Quality, Impact) and their AI Measurement Framework (Utilization, Impact, Cost).

One caveat worth stating: the sample skews toward organizations mature enough to invest in productivity measurement. The real industry picture is probably worse.

Now, the findings.

PR inflation: speed is eating discipline

The report states that PR size nearly doubled over the year, a pattern it describes as “AI-driven inflation rather than disciplined, testable code.”

Read that again. Median PR size went from 42 to 72 lines of code in four quarters.

Press enter or click to view image in full size

This is exactly what we’ve been saying for more than ten years: trust and quality must come before speed. The AI race is so intense right now that teams are bypassing principles we spent a decade establishing. Small batches. Testable increments. Single-purpose changes. All of it is quietly eroding because the agent can generate a complete feature in one shot, so why bother slicing?

Here’s why you should bother. When humans developed features, we naturally cut them into small, deliverable pieces. That slicing gave us control and deep knowledge of what shipped. We could test and experiment on each increment. With AI, the first output is already “complete,” so people are tempted to enlarge what they ask for. The model over-delivers. Entropy grows. And confidence in the generated code drops, precisely because the PRs are bigger.

The data confirms the loop: incremental delivery is the single sharpest decline among all developer experience factors this quarter.

Maintainable code that nobody trusts

Press enter or click to view image in full size

DX State of AI Impact in Engineering: Q2 Report

The most fascinating tension in the report is this one: code maintainability improved by 3.8%, while change confidence fell by 6.1%. As DX puts it, these two metrics are “historically correlated, but are now in tension.”

AI helps engineers understand the code in front of them. But faith in what actually ships is slipping.

My reading: given the sheer volume of code now produced (52.7% of all code is AI-authored, up from 19% a year ago), humans simply can’t stay in the loop anymore. Reviewing every diff at generation speed is physically impossible. So the human becomes the bottleneck, and under pressure, the human exits the inner loop.

Press enter or click to view image in full size

This is not a bug of the transition. It’s the transition itself. It’s the same conclusion pushed by a growing number of voices in the industry: the developer’s place is no longer inside the inner loop, line by line. It’s on the outer loop. Defining intent, designing the verification system, owning the gates, carrying the responsibility for what ships. If you don’t build that outer loop deliberately, you get the worst of both worlds: humans who no longer read the code, and no system that reads it for them.

The slow teams are winning on quality

Here’s the counterintuitive finding leaders should sit with. Traditional industries and Financial Services, the slowest-moving segments on throughput, report the highest perceived software quality. Financial Services posts around 77 to 81% perceived quality with the lowest throughput of all verticals. Healthcare, this quarter’s throughput leader, sits at the lower end of the quality range.

Speed and perceived quality are not moving together. And leaders should stop assuming they will.

This confirms something important: the companies extracting the most from AI are not the ones generating the most code. Some organizations go slower but put governance and a proper harness in place to control the quality of what gets produced. The report itself notes that structured rollouts are positively associated with success in AI initiatives. Those teams will likely end up with the better return on investment, because they can actually trust and build on what they ship.

Velocity without a harness isn’t acceleration. The report says it bluntly: some teams are simply shipping defects faster.

Press enter or click to view image in full size

The big one: spend is outrunning the return

This is the point that has evolved the most in recent months, and the one that will dominate every budget conversation in the second half of 2026.

The numbers are brutal. Median quarterly AI spend grew from roughly $1.5K to $44K in a single year. In the Tech sector alone, that’s about a 28x increase. Per-developer spend jumped from $9 to $191 per month.

Press enter or click to view image in full size

DX State of AI Impact in Engineering: Q2 Report

And what did that buy? The innovation ratio, the share of engineering time spent creating new features versus maintenance, stayed essentially flat the entire year, moving from 57% to 58%.

Time savings are real and growing, now over 6 hours per week on average. But those saved hours are not visibly converting into new value creation at the portfolio level. They’re being absorbed by the same organizational friction AI was supposed to eliminate: reviews, approvals, CI wait times, meetings.

Even more uncomfortable for enterprise buyers: smaller organizations pay the highest price per seat and extract the most throughput per dollar. Bigger AI budgets don’t guarantee better returns, and may even come with diminishing ones.

Stop counting tokens. Start counting saved weeks.

The report nails the instinct problem: leaders reach for new metrics like token counts and lines of AI-generated code, but these metrics only show how work is changing, not whether outcomes are improving.

I’ve lived this, and it’s what I keep pushing to teams. The accounting is backwards.

Don’t look at a task and say “this cost me $300 in tokens, that’s expensive.” Flip it. Start from the value: this feature was mandatory to win market share, or to improve the stability of the product. Without AI, it would have taken me two weeks. With AI, it took three days.

Now, and this is the critical condition, if I have a solid system with a proper harness, and I can trust what was produced as if I had built it myself over those two weeks, then the math is simple. I saved eleven days of development. Eleven days I can now invest in other high-value work. Whatever I paid in tokens for that task, the ROI is massively positive.

That’s how you should calculate it. And that’s how you should present it to your leadership. Token spend without a value denominator is noise. Token spend divided by trusted, delivered outcomes is a business case.

Notice the load-bearing word in that reasoning: trust. Without the harness, without the gates, without the governance that lets you treat AI output as if it were your own two weeks of work, the eleven saved days are fiction. You’ll pay them back later, with interest, in debugging and rework.

Think in systems, not in PRs

Uma Namasivayam from Dropbox asks the right question in the report: if your code throughput tripled tomorrow, could your SDLC absorb it? Their early signs say no. In his words, “it breaks everywhere.”

Human review. Code review. CI pipelines. Testing. Production visibility. Every downstream gate is cracking under generation pressure.

Press enter or click to view image in full size

The real mistake is to focus on code generation and PR production instead of thinking about the complete system. If you 10x one link of the chain and leave the others untouched, you haven’t gained productivity. You’ve manufactured bottlenecks. It’s far better to incrementally improve every part of the system, generation, review, testing, deployment, observability, to raise total throughput, than to over-optimize one stage and starve the rest.

The report’s data backs this everywhere: deployment frequency is up double digits, but review turnaround, local iteration speed and ease of delivery all declined. The overall Developer Experience Index slipped from 67 to 65. Individual output surges, the system absorbs the slack.

What leaders should actually do

The report’s prescription aligns with everything we advocate. While deployment frequencies increased across most segments, change failure rates remain highly volatile, with outliers now reaching plus or minus 3 percentage points against a 4% industry benchmark. Some teams are shipping defects proportionally faster.

The answer is to scale your quality infrastructure to match code generation speed: AI-driven testing, standardized prompts, updated code review protocols. I’d add: explicit gates, shared engineering standards that agents can consume, and observability on what your agents actually do.

Pair every speed metric with a quality counterweight. Deploy frequency with change failure rate. Throughput with change confidence. Time savings with innovation ratio. If one side of the pair moves and the other doesn’t, you don’t have acceleration. You have a system in tension, and it will resolve itself at your expense.

The starting line, not the finish line

The organizations best positioned for the rest of 2026 are the ones treating AI adoption as the starting line, not the finish line.

Adoption is done. 90%+ of the industry is there. What separates teams now is everything around the model: the harness, the governance, the verification loops, the measurement frameworks that connect tokens to outcomes.

The bottleneck moved from writing code to trusting code. The teams that build for trust will collect the ROI everyone else is still trying to justify.

Based on the State of AI Impact in Engineering: Q2 Report by the DX Research Team, published July 2026, covering 500+ engineering organizations from April to June 2026.

• Level 0, Gated: AI access locked down, restricted models, heavy approval processes.
• Level 1, Assisted: one engineer, one agent, a fast pair programmer. You review almost every change.
• Level 2, Parallel: one engineer orchestrating 5 to 10 agents, each checking its own work through tests, build, lint and security scans. You review final diffs, not keystrokes.
• Level 3, Supervised autonomy: around 100 agents, the engineer becomes a “manager of managers” steering an org tree of agents.
• Level 4, AI-native: 1,000+ agents, the loop is fully closed, most agents are kicked off by Claude itself, and the human steers “by intent”, monitoring by exception.

The framework is well built, genuinely useful as a map, and I recommend reading it. But I spend my weeks inside engineering teams, from startups to organizations with hundreds of developers, and there is a huge gap between this article and what I actually see on the ground. That gap is not neutral. It shapes expectations, especially on the management side. So here is my field perspective, and where I think we need to push back.

Where teams actually are, summer 2026

Just a few months ago, many companies were still at level 0: gated access, bridled models, endless security and approval loops. That phase is ending fast.

Today, the vast majority of teams I meet are at level 1. One developer, one agent, accelerated pair programming. Agents assist with the daily tasks, and almost everything they produce gets read before it merges.

And that is exactly why most organizations are not yet seeing real productivity gains at team level. Not because the models are not capable. Because the trust is not there. We have not yet built the infrastructure that would let us stop inspecting every output and let agents work without permanent supervision. As long as a human has to read everything, the human remains the bottleneck, and the “productivity revolution” stays theoretical.

The real transition happening now: level 2, the harness

The migration that actually matters today is the move from level 1 to level 2: building the harness and the guardrails that let a team gain confidence in the code being produced. Concretely, that means:

• precise skills and instruction files that encode the team’s standards and context
• custom linters and systematic typechecking
• solid test suites, including end-to-end verification in a real environment
• automated code reviews and automated security reviews

Once that foundation exists, you can run several agents on several tasks in parallel without supervising everything that happens. Your job shifts: instead of controlling each output, you control the complete system, and you continuously improve it yourself. That is the real unlock.

Here is the part I find funny: none of this is new. This is exactly what we have been trying to build for years, long before AI. CI pipelines, test coverage, linting, review processes, quality gates: it has always been about one thing, building a system that gives us confidence in the code being produced. Whether that code comes from humans or from agents changes nothing to the principle. AI does not change the nature of the problem. It changes its intensity, because the volume of code explodes and the cost of blind trust explodes with it.

The harness is not a project. It is a loop.

One thing must be clear, though: building this harness is a real investment, and it is never finished. A harness you set up once and forget becomes obsolete in weeks: the codebase evolves, the practices evolve, the models evolve.

The teams that get real value out of it treat it as a learning loop. Every agent execution is a source of signal. Every difficulty an agent ran into, every issue flagged in a PR or a code review, everything that “was not right” in the output gets extracted, analyzed, and turned into a new guardrail: an updated instruction, a sharper skill, a new lint rule, an additional automated check. The harness improves at every iteration, fed by the very work it supervises.

That continuous loop, capturing what went wrong and encoding it back into the system, is where the compounding effect actually comes from. Not from adding more agents. From making the system smarter every time the agents run.

Where I disagree with Boris

1. Context and skills do not arrive at level 3. They are fundamental from level 1.

In his table, encoding standards through CLAUDE.md and Skills only shows up in the level 3 guardrails. My experience on the ground is the opposite. Most teams have understood very early, often from their very first serious usage, how decisive the right context is. Instruction files, coding standards, architectural conventions made explicit: this is frequently the first thing teams structure, precisely because they see immediately that an agent without context produces plausible but wrong code. Context engineering is not an advanced practice for mature organizations. It is the entry ticket.

2. The whole framework carries a productivist vision: produce, even overproduce, at all costs.

Each level is framed around output multiplication. More agents, more parallel streams, more PRs, more tokens. And level 4, with its 1,000+ agents per person steered “by intent”, is implicitly positioned as the destination everyone should aim for.

I think most organizations are simply not capable of operating at that level today, and it is not a skills problem. Individually, some engineers could probably handle orchestrating that. Organizationally, it is far too complex. We do not have enough specifications. We do not have business needs formalized to the point where an agent can pick up work autonomously and get it right. We do not have enough downstream control, enough test coverage, enough deployment capacity toward real users to absorb that volume of change safely.

Level 4 fits very specific contexts. A team of ten superstars building a startup in Silicon Valley, with a young codebase, no legacy, no regulatory constraints, and full ownership of their product? Yes, absolutely. An enterprise with hundreds of developers, decades of legacy, strong business and compliance constraints, and multiple teams sharing responsibility for the same systems? No. Presenting 1,000+ agents per person as the absolute goal for everyone is, frankly, disconnected from reality.

3. This article needs to be read for what it is.

Boris is brilliant and Claude Code is an excellent product. But he is also there to sell tokens and to maximize the use of Claude. That is not an attack, it is literally his job. It simply means that his maturity scale is also a sales funnel: every step up the ladder multiplies token consumption by an order of magnitude.

There is nothing wrong with a vendor publishing a vendor’s vision. The problem starts when we forget it is one.

The manager problem

Because here is what actually worries me. The people held up as stars in these narratives, shipping dozens of PRs a day, steering fleets of agents from their phone, are presented as the teams we should all aspire to become. And on the management side, this kind of discourse quickly becomes “the reality of the market”.

Managers read these articles. They see the level 4 column. And they start pushing their teams in that direction, without asking whether the context fits, without measuring whether the harness exists, without checking whether the organization can absorb the output. The narrative becomes the roadmap. And that is how you push teams into mistakes.

What the data actually says

We already have evidence for what happens when AI adoption outpaces system mastery. The DORA studies show it clearly: at the level of complete teams, AI-related productivity gains remain modest, and adoption correlates with increased instability of the systems being delivered. More code, produced faster, does not translate into more value shipped.

Press enter or click to view image in full size

DORA report — 2025

And the second-order effects are just as real. Pushing everyone to produce more, every day, to live in permanent hyper-efficiency, tends to disorder systems and inject entropy into codebases. It degrades the daily experience of developers. It creates stress and cognitive debt: people reviewing six streams of generated code they did not write, all day long, under pressure to go faster.

The end of that chain is predictable. We will produce more code and more features, and we will also produce more bugs in production. And from the point of view of the end users, the only people who actually matter, we will have degraded our product and our image. Nobody outside our industry celebrates PR counts.

Press enter or click to view image in full size

An AI agent relying on outdated documentation caused a major outage at Amazon

We are not in a token race

Let’s say it plainly: the goal is not to consume as many tokens as possible, nor to run as many agents in parallel as possible. Token consumption has never been an indicator of value. Burning tokens on work nobody needed, on code that gets thrown away, on agents supervising agents supervising noise, is not maturity. It is waste with better marketing.

The same goes for our existing practices. The quality processes, the team workflows, the communication between testers, product and developers, the constraints we spent years putting in place: none of that is bureaucracy waiting to be automated away. These are guardrails we built deliberately, precisely so that our systems do not do just anything. Reaching a level where Claude picks up its own tasks, anticipates everything and does everything in our place means breaking many of those constraints. That is not a productivity unlock. That is a loss of control over what gets created underneath us, dressed up as progress.

My conviction: aim for level 2, and in some contexts level 3

Level 2, and for some teams level 3, is where the sweet spot sits today: the middle ground between quality and speed. You genuinely multiply your capacity, you parallelize real work, and you keep control of what is produced, because you own and understand the system that produces it.

And let’s be honest about the effort involved. Building that complete harness coherently, the instructions, the context, the tests, the automated security checks, the automated code reviews, will take months, sometimes years, for most organizations. And it does not end there: the harness has to be maintained and improved continuously, fed by what every agent run teaches you. That is normal. It took us years to build our current engineering practices too.

Which is exactly why skipping steps is so dangerous. If we burn through the levels because a vendor shows us his own usage as the norm, many companies will drive straight into the wall: unstable systems, exhausted teams, production incidents, and users paying the price.

It depends on your context. It always did.

Maybe, much later, level 4 becomes accessible to ordinary organizations. Maybe the specifications, the controls, the deployment maturity will catch up. But that is not where we are, and pretending otherwise pushes teams into mistakes today.

As always, everything depends on your context. And in the vast majority of contexts, you do not need 1,000 agents per person. You need trust in what your agents produce, a harness you master, and a pace your organization and your users can actually absorb.

So let’s not put that pressure on ourselves. Let’s build trust, one step at a time. That is what real maturity looks like.