You can generate a million lines of code in a day now, but working out whether any of it is right has become the expensive part.
By Dean McPherson, co-founder of Paperform
Every developer I know has had the same year. The part of the job that used to take longest, sitting down and actually writing the thing, now takes almost no time at all. Writing code has stopped being the bottleneck in building software.
But bottlenecks don't disappear when you widen them, they move somewhere else. And unfortunately, the new bottleneck has moved somewhere much less comfortable. You could generate millions of lines of code today, but you won’t have much faith that any of it works. The new bottleneck is whether you can trust what just got built.
The abstraction is the appeal, and it's also the problem
AI is extraordinarily good at taking something complicated, hiding the machinery, and handing back a clean answer. That's the whole appeal, and it's why it feels like magic. You ask for a thing, the thing appears, and you never have to know about the fifteen decisions that got made along the way.
For plenty of work that's completely fine. Throwaway scripts and first drafts you were going to rewrite anyway don’t need an audit trail.
But that trade gets steadily worse as the work gets more important, because the same abstraction that saved you the effort has also removed your ability to check. You cannot verify what you cannot see or understand, and once you cannot verify it you have stopped being in control and started hoping.
There's something genuinely new in that. Software has spent decades offering people easier tools in exchange for less power, and this is the first time we can remember a tool offering more power in exchange for less visibility.
Meta could have built a chat app
At the start of September, Meta told its staff it was moving internal communications to Slack. In the memo, Meta's AI chief Alexandr Wang pointed to Slack's developer tooling and the integrations already plugged into it, and said it was the strongest platform for AI agents.
Meta employs some of the best engineers alive, and it had already built its own workplace chat tool. They ran that tool for years.
But they bought one instead, and that is very telling. They know that the real cost isn’t building. It’s maintaining, securing, and supporting everything around it indefinitely. It’s about not needing to create and maintain systems to test and verify what you have built.
We’ve faced the same choice across our stack. AI may make the first five minutes of building feel free, but it hasn’t changed the long-term cost of owning a production system outside your core business. That “quick build” can become a decade of maintenance.
So where did the work go
Even with the speed AI enables, software needs the same amount of care it always did. The reality is that, to the non-technical user, AI “hides” how something got made. This moves where that care has to be applied. The attention that used to go into writing code properly now goes into establishing, after the fact, that the code AI wrote can be relied on. This process is slower and it happens after you've already committed to the build.
Which means the work has moved into systems.
Tests that catch real failures, ai reviews of AI reviews of AI reviews…. Hoping for some way of demonstrating afterwards that the thing did what you hope it did.
That's hard and exhausting.
What you get in the end is output from a very sophisticated token generator that you need to proof. It is often brilliant. It is also completely incapable of being accountable for itself. So guess what that means? Accountability stays exactly where it has always been; with a person who has to answer for it.
Don't get us wrong. None of this is an argument against building with AI. We use it heavily, every day, and it has made this the nimblest our team has ever been. But we need to recognise that AI has relocated effort, not erased it.
Operators aren’t always developers
Everything above assumes you're technical enough to attempt verification at all. Most of the people we build for aren't, and we don't think they should have to be. We previously wrote about the gap AI opened between those two groups, and this is that same gap seen from a different angle.
What non-technical people need is control, though control here means something different from what it means to a developer. For a developer it means the ability to do powerful things, things that they understand because they built them. An operator needs something different. They need to see what happened, change it when it's wrong, and understand enough of it to be comfortable putting their name to it. They need verification.
Give someone that control and what you get back is relief. Relief is badly underrated in business software. It doesn't demo well and it doesn't screenshot, so it never makes it onto a pricing page. But it's still most of what people are actually paying for. Building something powerful that doesn't make the person using it feel small is arguably the hardest design problem to solve.
The question worth asking
If you're weighing up a tool at the moment, or weighing up building your own, the usual question is what it can do. We think it’s critical to ask something else first. If this tool, build, or system gets it wrong, how will I find out?
It's a duller question, but it sorts things quickly. If you'll find out straight away and can fix it yourself, go and build, and enjoy it. If you'll find out in three months because a customer told you, then whatever you saved on the build has moved somewhere harder to see, and it's still sitting on your books and costing you money.
We think the industry has been slow to say this plainly. AI will generate the work. It can't carry the responsibility for the work. That stays with whoever has to answer the urgent chat or call. Every tooling decision from here is really a decision about who is holding that, and about how much they'll be able to see when they need to.
For the operator who has been handed responsibility for a system without any of the means to inspect it, the discipline has to come from the software itself. And they miss it because capability demos well and relief doesn't.
Which is where the bottleneck has come to rest. The bottleneck now sits with the number of people who can look at what a machine produced and say, with any real confidence, that it's fine. It's the one part of this you can't widen by buying more tokens.
Our bet is that this is where the next few years get decided for many tools. What tools can do is converging fast and will keep converging. What will separate them is whether the person who's accountable for them can tell what happened, and whether they can put it right without asking anyone's permission.
Nobody has ever felt relief about software they didn't trust.
