You’re looking at a list of things you’d love to do, and you’re looking at AI coding tools as a way to boost your way down that list. You might not have the relationships mapped out, but you can see there is some route to value if you spend on LLMs that speed up code.
You’re now in the developer productivity game.
The idea behind developer productivity
The roots of developer productivity are straightforward. Some smart engineering managers figured out that a small team of developers with the best machines, screens, and development tools could generate value at a rate and quality that far exceeded their “head count”. You could also supply all these upgrades to developers at a cost way below the fully loaded cost of 1 more developer.
The return on investment for this approach was incredible, but traditional engineering managers didn’t understand it. They thought developers were asking for more screens because it made them look more important. This emerged from organizations that rewarded managers for empire-building by granting them larger offices with better views.
I’m a big fan of Ron Westrum’s Typology of Organizational Cultures. For this post, though, we’ll keep things simple and refer to traditional thinking (keep equipment costs low) and modern thinking (provide high-quality tools).
We have never shaken off this traditional-versus-modern divide over developer productivity. And now, the subject has returned to the spotlight due to AI and, more specifically, LLM-based coding tools. Your organization’s past approach to developer productivity will determine whether you can successfully integrate AI tools into your development teams.
Let’s look at why.
A tale of two cities
Traditional organizations operate through control. Managers dictate how work is done, choosing the processes and tools workers must use. Instructions flow downward, and managers define efficiency. Workers are evaluated individually against the manager’s prescribed methods, rather than by outcomes.
Modern organizations operate through trust. Teams choose how to work, selecting from available options or proposing new tools when needs emerge. Authority flows to those closest to the work. Performance is a team sport measured by outcomes.
| Traditional | Modern |
|---|---|
| Operates through control | Operates through trust |
| Efficiency is manager directed | Productivity is worker-led |
| Finds the cheapest tools | Chooses the best tools for each job |
| Prefers expanding teams | Prefers small teams |
| Tooling is a cost | Tooling increases value |
| Performance is individual | Value flows from collaboration |
As we experiment with AI coding tools, we are gaining crucial insights. We are developing a better understanding of how much human oversight is needed to successfully and sustainably deliver high-quality software to users. A strong guiding hand is crucial in directing and correcting the output of these tools.
The value of any software you build, by hand or with assistance, comes from the flow of information. That means listening to software users and collaborating internally. While the code is what gets left behind by the process, it’s only an artifact of a more fundamental learning process. The ability to learn and share knowledge will also benefit teams as they discover how to apply AI coding tools to this process.
It’s also clear that Continuous Delivery and automation remain paramount. In the past, automated linting, security scanning, and tests gave us confidence in the code teams wrote; now, they can provide us with confidence in code generated by LLMs. DORA’s AI Capabilities Model includes 7 capabilities essential to successful AI adoption, including user-centric focus, strong version control practices, and working in small batches.
For organizations that haven’t adopted Continuous Delivery, rocky shores lie ahead when they unleash AI tools on their codebases.
Return on an unspecified investment
Now here’s the fascinating conundrum for anyone trying to calculate a return on investment for AI coding tools. The commercial tools indeed remove many tasks that, as a developer, I don’t want to do, though they also introduce new ones. I see why people would like to use them to remove much of the noise and focus on the essential details in the software. The problem is, you run out of credits fast, so if you want to use these tools full-time, you’ll need subscription levels that support that.
Credit exhaustion is the first friction point where traditional organizations will come unstuck. Developers who rely heavily on AI coding tools will slow drastically when credits run out. This will likely become a significant problem over time as developers become more dependent on working at the high level of abstraction that prompting offers.
Imagine if coding languages had similar limits. You’d run out of Python hours and have to continue your work using assembly language.
Organizations with a cost focus will challenge developers who want a higher budget for these tools. Any manager who has previously denied more screen real estate is likely to reject higher subscription costs for AI coding tools. Their view in both cases is that the promised productivity isn’t real.
The second hurdle for these commercial tools is the uncertain future pricing. We know some AI companies are burning through investment cash, which means the price we pay is subsidized by their desire for growth. There must be a pivot point at which they begin the search for profitability. This will once again trigger problems in cost-focused organizations.
Some developers are already thinking ahead and looking for open-source models they can run locally to reduce cost uncertainty, but, as always, you pay one way or another. The time spent assessing, updating, and managing these models is a direct loss of the productivity you’re trying to gain.
One solution may be for commercial vendors to offer fixed-price, unlimited use through local models. The challenger to this solution will come from Platform Engineering or DevEx teams, who could supply a packaged open-source local solution for developers to reduce the overhead of selection and maintenance.
The nature of problems changes
Traditional and modern organizations face the same challenges, but you can see that culture fundamentally shapes how they are addressed.
Modern organizations will judge their return on investment by the value they deliver. Their past investments in Continuous Delivery will provide a solid foundation for them to experiment with new tools, and they’ll creatively address the cost issues associated with AI coding tools.
Traditional organizations will seek to minimize costs, avoid investing in automated pipelines, and demand higher developer output with no real basis for expecting it.
The set of capabilities a modern organization applies to high-throughput, high-quality software delivery is surrounded by subtle, interconnected relationships. For the traditional organizations that just want to “buy AI”, the benefits are unlikely to arrive.
Happy deployments!
Tags:
