← Blog

AI has broken our software metrics.

In 2026, Amount of code produced doesn’t mean what you think it means.

There is a lot of chaos in the Software Engineering industry right now. Every day we hear about new layoffs, new hype around AI coding and new promises of what buying more AI tools will get you.

For developers they land weakly, but for managers and executives, they sound like candy. In my last article I talked about why companies are not seeing the ROI that the AI labs keep promising. For the moment, I’m going to set aside the institutional damage done downstream of that broken promise, there are plenty of other articles on that topic. There’s something much more interesting to look at here, I think.

I’m a systems thinker. When I see a phenomena I want to understand WHY it is occuring, separate from opinions and pre-conceived beliefs… and I’ve been working on this for the last few months and I’ve reached an interesting conclusion which I will share with you today.

What are we seeing in the world of software development?

So… Let’s start with the state of things. What are we seeing in software development today?

We are seeing, in no particular order:

I won’t go into detail disecting each of these… They are all occuring and the causes are so tied up in the organizations that it’s practically impossible to make wide statements about them. There is an underlying factor, however, and it’s actually quite surprising.

Get on with it. Tell me the secret.

Well, it’s actually quite simple and difficult to unsee once you see it. For decades we have used ‘Amount of Code Produced’ (AOCP) as the proxy-handle for judging the effectiveness of the development team. This is so much of a ‘no brainer’ that I heard your collective ‘what else would you use’ as I sit here.

Before I continue, let me start by pointing out the terms you will hear related to AOCP: First there’s the old standby ‘lines of code.’ It’s been rebranded as ‘number of commits’ or ‘Number of PRs.’ or ‘PR size’ or ‘code volume’ etc. All of these are thrown around in the AI marketing and following discussion.

I’m here to tell you that AI code generation has completely destroyed AOCP as a measure of engineering performance.

The question is: Why?

The answer is simple, before AI coding, code produced was what occurred after EVERYTHING ELSE in the development process had been completed. It couldn’t really be done prior to the developer understanding the full context of what was being requested. Business intent, edge cases, stakeholder negotiation, system modeling, architectural evaluation, integration discovery, etc. etc. In short, an understanding of what needed to be done in all it’s detail and nuance.

This is specifically why it was a good proxy when humans wrote the code. If a human produced working code, it meant the previous steps all were completed at least satisfactorily.

The wrench in the works: AI coding

I’m not one to deny the impact of AI coding. With AI coding, you can create code in minutes that used to take hours. And herein lies the problem, because AI doesn’t require a full understanding of context to produce code. AI agents and tools can produce prodigious amounts of code very quickly with very little input. The trouble is not that the code is of low quality (even though it often is) or contains bugs (even though it often does)… These are things that can be rectified… sometimes even by the AI in question.

The trouble we have right now is that the coding no longer requires the completion of the previous steps. You can have an AI coding agent writing code as quickly as you can type ‘add commenting as a new feature.’ And if you are measuring results based on AOCP… it will look like winning.

That said, real software is more nuanced, there are 1000 questions that need to be answered to integrate a feature into an existing codebase. Who can comment? On what? Are they visible immediately or do they need to be approved? Who can delete them? Can people reply? Can users upvote/downvote? Who can see comments? What content guards do we need in place? Is this public or private only? Are the users likely to be well behaved (staff members) or do we need to deal with bots and malicious activity?

This is just a sample on an extremely simple request. Can The AI build the code if it has all those answers? Yes… and there’s a good chance it will be as good or maybe even better than the developer on staff would produce. However, this is the crux of the problem.

All but the most green professional developer would not start writing code until he or she had most of those answers… because they know revision after the fact will take much more time than doing it correctly in the first place. AI coding agents are not so burdened. They will happily write code, making guesses or ignoring important details as soon as you ask them to start.

That’s bad, right? Possibly. It would certainly be hard for a human, because revising code when the additional details are discovered is a harder task than writing it in the first place. AI coding agents don’t have that problem. They will happily regenerate the entire code using new information when it is provided. And they will do this repeatedly as often as you ask. You know what that’s good for? our Amount of Code Produced (AOCP) metric. How does it show up? Multiple Pull Requests. Our 5 revision PRs look AMAZING on the AOCP dashboard. The trouble is that we took 5 PRs to get the code approximately correct, where a developer who had the context required could have done it in 1.

And I’m not just making this up: Faros AI observed that high AI-adoption teams merged 98% more pull requests and completed 21% more tasks. Individual throughput metrics skyrocketed because AI made issuing, re-issuing, and updating PRs practically free. Twice as many PRs, 1.2x as much done.

A human developer spending time upfront to get context delivers the code in 1 PR. Generating 5 revision PRs forces the surrounding engineering system to absorb the trial-and-error process.

It’s tempting to ask which one is the right way? I’d argue that’s a distraction. They are two ways to approach the same goal. There are impacts, 5 PRs take 5x the time to review of one, for example… but regardless there ARE a number of questions that fall out of the situation. For a business, some useful questions are:

You know what doesn’t make that list ‘How much code was produced?’

Is that bad? No. But it breaks something fundamental in how we’ve been evaluating software development for decades.

Amount of Code Produced no longer correlates to benefit produced to the company

Or to put it another way. when AI can produce 5x as much code to get the same task done, it doesn’t matter if the last one was correct, your correlation between AOCP and benefit is broken.

We are measuring the quality of our commute by how fast we can go and how many miles of road we took to get there. You can drive 75mph on the highway, back and forth 5 times to go 1 mile… but we all know that doesn’t make your commute better.

How this causes broken engineering organizations.

Ok… so Amount of Code Produced doesn’t work as a metric anymore, what’s the big deal? This is where the conversation about AI coding gets muddled and broken in the organization. And this requires a real inspection because it’s tempting for people, engineers especially, to just say ‘Mangement doesn’t understand Software Development’ and stop there.

But the truth is more subtle, and this is why the conversation around AI coding is so broken. It’s not that management doesn’t understand software development, though that is certainly true and understandably so. Software engineering is a highly skilled and specifialized activity, no one who doesn’t do it fully understands it. That’s not the problem. The problem is that in the age of Human Programmers, Amount of Code Produced was a perfectly usable proxy for progress.

As I established earlier, that proxy is broken by AI coding. The trouble is that the tool that management has been using for years to judge progress suddenly no longer works.

And this is why many managers and executives are frustrated by the state of their engineering organizations right now: Because AI vendors are saying ‘you can produce 10x more code in the same time’ - and they aren’t wrong. That sounds like ‘You can get 10x the number of features’ to anyone used to using AOCP as a metric. And so management mandates AI usage, and gets exactly what the AI vendors promised… but not the ‘10x number of features’. Which, it turns out, is what they actually care about.

This can create a huge amount of conflict in engineering organizations, because the engineers really can be doing 10x more coding… it’s just not delivering the effect implied by the AI vendors. It CHANGED what they are spending their time on, it didn’t do the same thing in less time.

And managers are left trying to figure out why the AOCP metric claimed by the AI vendors isn’t working for them. All too often, the conclusion they arrive at is ‘The team must be resisting AI usage.’ Add to that that engineers are often acutely aware of what it doesn’t help with and that reinforces that conclusion. Now you have distrust on both sides. I’ve seen engineering organizations destroyed by exactly this dynamic.

As long as we try to use AOCP in the age of AI coding, we will continue to have this conflict and these missed expectations.

Ok. It’s all broken. So what do we do now?

This is the hard part. We’ve been using Amount of Code Produced (AOCP) for so long as a standin for progress that even now that we know it’s broken, it’s difficult to imagine anything else. Nevertheless, we have to find something new to represent our progress as a software engineering organization. And it’s going to be a bit more difficult, because it actually requires us to be concrete about what we care about.

That said, I think we can break it down into a few broad categories, which I will provide below with some examples of each. These are not exhaustive, but point the way.

Ultimately, these together stand in for what we used to measure with AOCP: How much value are we getting for the effort/cost expended?

Why this might actually fix things.

There is a ray of hope in all of this that you might not notice at first. When management shifts away from AOCP to the core metrics above, an interesting thing happens. You suddenly have management and engineering aligned. Every software engineer wants to build features efficiently. Every engineer wants the system to work right and not wake them up at 2am on Saturday to fix production bugs. Every engineer wants their code to work without multiple code review sessions. Anything that works against those things is suddenly a priority for both management and engineering at the same time.

What a world we are suddenly living in! When management and engineering are looking at the same scoreboard with the same goals, amazing things can happen.

If your company could use help with this shift, Ionzero can help.

© 2026 Ionzero Inc.