Sometimes it feels like there are as many modifiers on “Delivery” as there are on a fancy coffee order, and why do they all have to start with C?
I’m here to add another to the list.
What is Controlled Delivery?
Controlled Delivery is the set of practices that ensure you’re delivering valuable changes to software behavior, not just detonating a change across an entire ecosystem. The goal of controlled delivery is to get the value of a change to the person who benefits from it. This is not about the change itself, because sometimes a change is not the point. Thinking of change as value is a mistake we all make, because that’s what we’re evaluated on, but it’s not what matters to the user.
Here are the things that we think make up Controlled Delivery
Composable content
Software isn’t a one-way flow from the maker to the user, it’s a collaborative work object, and it’s only a small part of the user’s day. As software makers, we need to give users the option to choose the parts that add value to their day and leave aside the parts that they neither need nor want. That means that our concepts of modularity and microservices can’t stay locked inside our own architecture, they have to be obvious and manipulable to the user. That’s why APIs, and now MCPs, are such a powerful concept.
From a sales and adoption perspective, of course it would be great if people used all our features all the time, but that’s not very realistic. I’d bet that you’re using less than 20% of the available features in your daily software, and still getting good value from it. And probably, you’d have an easier time using it if there were fewer options available.
Composable content is what Jordan Mallory is asking for in this 2020 post.
Big video games require a lot of expensive hardware to run them, and dedicated time to play them. Stardew Valley was a hugely successful game with objectively rudimentary graphics and skills requirements, exactly because anyone could get rewarding play out of it, even on an old cell phone.
If you’re reading articles about CI/CD, you’re probably not One Guy (™) working at a table in the corner of your bedroom, but the point remains - the value is not in what you deliver, but in what the user can do.
Mindful rollout
Change, especially rapid change, is hard for people to deal with. Alvin Toffler wrote Future Shock in 1965-1970, and it was about how much people were struggling to adapt to the rate of change they were facing. That problem is not, I’m sure you’ve noticed, going away.
That means that whenever we deliver software to our users, we’re asking them to accept change, which they don’t like. We owe it to them to make sure it’s a change that benefits them, and that it’s a good delivery.
We have a lot of very good evidence that deploying rapidly/continuously makes software development safer and more robust. Let’s keep doing that. But as a user, I would much rather have all those changes packaged up into a coherent and periodic bundle, so I only have to get my process disrupted once in a while.
How do we make sure what we’re sending out is good? We want our product to be
- Safe
- Compliant
- Robust
- Valuable
It needs to not create new problems, it needs to fit in with the existing software, it needs to operate at the scales it will face, and it needs to be a net addition to their workflow.
We use feature flags in a variety of patterns to accomplish these goals.
- To make it safe, we do staged rollout or ring deployment to test on subsets of users before we push it to everyone. We also include an andon cord in case a deployment should be stopped and rolled forward.
- To make it compliant, we make sure our deployment tests include legacy users and people on old hardware, as well as early adopters
- To make it robust, we run shadow versions in parallel with the visible versions and blue/green overlaps
- To make it valuable, we collect feedback on who uses new features, and communicate directly with them on how it works and what they need.
Many of the tools of mindful rollout already exist and are being used for load evaluation, user entitlements, or A/B testing. What makes it truly mindful rollout is that the people doing the deployment are actively monitoring and watching the changes - not just to preserve operational stability, but to understand how and why people are using the changes.
Verifiable receipt
Everything that is deployed goes through the same deterministic steps, and is recorded and hashed. That means that every change has a signature that can be traced and affirmed. That signature carries through from the initial developer change to the release on the user’s machine. No surprises, no flex. Controlled Delivery has to include knowing what actually arrived, as well as what we intended to arrive.
What isn’t Controlled Delivery?
Controlled Delivery isn’t a compliance system in itself, although it helps you do compliance. Controlled Delivery is not an unwatched automation. Controlled Delivery is not an excuse to deploy all the time.
Controlled Delivery isn’t about the software creator’s experience, it’s about the software user’s experience.
How does that fit into CI/CD?
I know a bunch of you just flinched at the thought of having to do yet another process in your SDLC. Especially one that doesn’t have a track record of success or a cool conference talk (yet). But this is actually built out of parts you probably already have, just used in a different way. Ring deployment, trunk-based development, feature flags, signed changes, those are all mature technologies with over a decade of best practices behind them. When we say Controlled Delivery, we’re inviting you to combine them in a way that works for both you and your users.
What’s my action item?
First, decide if Controlled Delivery will help you deliver your software better and more effectively. If it doesn’t meet your goals, it’s not a good tool, no matter how nifty it seems.
Then survey your SDLC and see how many of these patterns you already have in place. What can you chain together to make a continuous, predictable, sensible flow from code change to user value? What runbooks, approval gates, automations, and release rituals already exist, and could they be incorporated into Controlled Delivery?
Finally, build or buy the parts you’re missing. I can’t tell you how that tradeoff will break down. Things you build are perfectly tuned for your current processes and people, but you’ll have to maintain and update them internally forever. Things you buy will require integration and adaptation, but will be maintained by the vendor.
Controlled Delivery is part of the seamless software motion we’ve all been working toward, and I hope you see the foundations in your own work.
