Continuous Delivery is a method that improves the speed of software delivery. Its adoption leads to higher deployment rates, continual process improvement, and increased team happiness. Simply, Continuous Delivery is the best way to deliver software we know.
Here we explore how the automobile industry helped shape Continuous Delivery’s ideas, plus the principles and capabilities you need to adopt it.
Why Continuous Delivery owes a lot to car manufacturing
Continuous Delivery borrows some ideas from the lean manufacturing production method, popularized by post-war Toyota.
Before the lean method, car manufacturing was rife with financial risk and waste due to industry-standard processes.
For example, it was common for manufacturers to:
- Build cars in huge batches for anticipated demand, so they always had stock ready for sale
- Build parts in large numbers to reduce unit costs
- Focus each manufacturing run on a single model to limit process changes
- Only check vehicles for quality at the end of the production line
On the surface, each practice might make sense. Dig deeper, however, and you find flaws and risks.
Yes, you could argue you need stock to sell stock. But what if demand isn’t what you thought it’d be? Unsold cars and unused parts take up space. Even worse, they might rust on your lot. In either case, you spent time and money building something for nothing.
Likewise, building one model at a time might feel the safest way to ensure quality. Yet while committed to a production run, you can’t produce other models you might have urgent demand for.
And lastly, and probably the most wasteful option on the list, is performing quality assurance at the production line’s end. By the time your inspectors find faults, cars are complete, and your teams not only wasted their time and effort, but also your materials.
By adopting lean, Toyota adjusted their processes so they:
- Worked to their actual demand, building only the cars and parts needed to complete orders
- Developed new processes to change machine setups quickly so they could react to order variety
- Empowered their staff to stop production and solve problems if they found problems from the station before
The result was less risk and fewer wasted hours and materials. Plus, particularly useful for moving the analogy over to DevOps, they found staff more engaged in their work.
Though Continuous Delivery isn’t a 1:1 of the lean manufacturing method, it’s what Toyota learned that’s important. Namely:
- Work small and to demand
- Improve your processes
- Check quality throughout
- Empower staff to solve problems
Software delivery experts have since evolved and expanded on these lessons. The result is 5 principles and 7 capabilities that will help you adopt Continuous Delivery.
Continuous Delivery principles
Continuous Delivery uses 5 simple principles to guide organizations on its adoption and help keep them focused in its use.
The 5 principles are:
- Build quality in
- Work in small batches
- Automate everything you can
- Always improve
- Everyone’s responsible
Let’s look at each Continuous Delivery principle and explore what it involves.
1: Build quality in
As with traditional manufacturing methods, software delivery usually places quality checks at the end of the production line.
When placed at the end, however, quality can become an afterthought. This means you might:
- Not realize there’s a problem with your delivery processes or software until the end of a sprint
- Waste time and effort on something that doesn’t work
- Repeat mistakes without realizing it
Instead, you should build quality into your deployment pipeline. This means designing your deployment pipeline so it detects faults during all phases. Aim to shorten the time between a problem’s introduction and its discovery.
Once found, immediately update your processes or systems to catch the fault earlier in your pipeline or remove it completely.
2: Work in small batches
Organizations not using Continuous Delivery try to limit downtime by releasing updates together in huge batches.
While this feels like a sensible and safe option, it instead causes a lot of problems:
- Developers must work against change windows and commit deadlines
- A single broken update can prevent the other changes in its batch from going live
- It’s harder to pinpoint which update breaks something
- Rollbacks could be a long process if you can’t fix it quickly
In comparison, by working in smaller sprints and batches instead, you will:
- Get faster feedback for your product and the effectiveness of your deployment pipeline
- Reduce the risk of an update breaking something
- Make rollbacks to a previous version much easier
- Increase the predictability of releases and your confidence in them
- Quicken each phase of your pipeline
3: Automate everything you can
Humans are not particularly great at performing repetitive tasks for a long time. People make mistakes. People get tired or bored. People get distracted.
Computers, however, are excellent at repetitive tasks and are capable of doing them as often as you need. And without coffee breaks! That’s why with Continuous Delivery, you should automate something if it’s possible to do so.
By automating, you free staff to work on things humans are great at, like problem solving, collaborating, and improving your processes.
If you’re about to start with Continuous Delivery, use established tooling to automate the following as soon as possible:
- Code compiling, tests, and merging
- Packaging
- As much of your deployment process as possible
This will give you a great starting point to make gains elsewhere in improving your deployment pipeline.
4: Always improve
Though automation is an important part of improving your processes, it’s not the only thing you should focus on. Plus, tasks already automated will have room for improvement.
In particular, look for unnecessary steps or those that slow updates down as they move through your deployment pipeline.
For example:
- Can you make an already automated process quicker?
- Can you automate or speed up provision of infrastructure or environments?
- If an approval step slows down your delivery, do you still need it? If so, can you automate the approval, either fully or partially?
- If something in your pipeline fails and doesn’t affect your software delivery, do you still need it?
- Is there tooling you haven’t considered that could improve or speed up your processes?
You should never stop asking these questions or striving to improve your deployment pipeline and its workflows.
5: Everyone’s responsible
Work culture is one of the most important things to address when it comes to adopting both DevOps and Continuous Delivery.
Continuous Delivery works best in what the Westrum Typology of Organizational Cultures calls a ‘generative’ culture.
To create a generative culture, you must encourage an environment where team members:
- Cooperate often and reject silos
- Share a common goal and responsibility for the product
- Feel comfortable enough to take risks
- Accept mistakes and failure as part of the process and do not play the blame game
- Feel safe to speak honestly and freely
Continuous Delivery capabilities
The 5 principles help guide decision making, but there’s also technology and processes you must have in place to adopt Continuous Delivery.
Specifically, there are 7 capabilities needed for a successful adoption:
- Continuous Integration (CI)
- Trunk-based development
- Continuous testing
- Monitoring and observability
- Loosely coupled architecture
- Database change management
- Deployment automation
These technical foundations allow you to build, perform, and refine your processes.
Let’s explore each.
1: Continuous Integration (CI)
Traditionally, developers worked to deadlines or only committed code once they’d finished a feature. Working this way gave build managers time to compile and test their work. This was a manual and methodical process that left developers waiting for feedback and slowed down software delivery.
Instead, Continuous Integration improves commit rates and automates time-intensive and repetitive build tasks.
Rather than waiting for deadlines, developers commit changes to the code repository many times throughout the day. As soon as they do, a build platform (also known as a ‘CI service’) automatically compiles and tests their changes.
If successful, the build platform can package the code for deployment (or trigger packaging). If the build fails its tests, developers know there’s a problem straight away.
The benefits are:
- It’s easier to find and fix problems in small commits
- Developers get almost instant feedback
- No one’s bound by deadlines or change freezes
- Automation’s repetition makes results more predictable
- Everyone gets more time - developers can code without barriers, operations teams can work on more important tasks
- New features move through your deployment pipeline much quicker and more reliably
2: Trunk-based development
Almost all software developers are familiar with ‘distributed development’ thanks to Git. And we wager most development teams already use Git repositories like GitHub or similar services.
Distributed development is hugely important in modern software delivery. It allows developers in large teams to work independently in their own branches before merging into the main source code.
This is great! It means everyone works safely and no one’s work affects anyone else. However, Continuous Delivery still recommends setting some limits with branching with trunk-based development.
Trunk-based development is when developers either:
- Commit code directly into the main branch
- Use 3 or fewer branches and merge into the main branch within 24 hours
The reason for these limits is the longer developers work on separate branches, the higher the risk of merging problems. By committing to the main branch as often as possible, you are:
- More likely to build on the newest code
- Less likely to see developers’ work clashing against each other when merging
3: Continuous testing
We talked about testing as part of Continuous Integration, but testing doesn’t end at a successful build. Testing happens throughout your deployment pipeline and in all environments, including:
- Usability
- User-access testing (UAT)
- Security
- Post-release vulnerabilities
One truth remains with all testing: It should happen often, quickly, and—if possible—automatically. Invest in testing tools to take the load off your teams.
The structure of testing is also important. Strategically run your fastest and automated tests first. That way, if you find a problem, you haven’t wasted effort on slower or manual tests and can use that time elsewhere.
4: Monitoring and observability
Much of operations revolves around tracking infrastructure stability and security. The goal is simple: Ensure services and systems work as expected so you can prevent outages.
Continuous Delivery expands that goal. You should collect data from every possible source and use it to spot all patterns and relationships, not just those affecting performance.
Continuous Delivery also recommends pulling all those metrics into a single tool. Doing so helps you spot relationships easier and react much faster. Plus, it can help direct your improvement efforts.
For example, you could see when a new version causes complaints by tracking software versions and support tickets in one place. Or, by tracking deployments and cloud infrastructure activity, you see the changes making your costs rise.
The data to track depends on your business needs. We recommend experimenting, though, as you might find unexpected but valuable insights.
At the very least, you want to track:
- Infrastructure performance
- Internal process times
- Application feature usage
- Business metrics
5: Loosely coupled architecture
‘Loosely coupled architecture’ is a development structure where teams build apps in separate components.
Each component has:
- Its own infrastructure
- Few dependencies
- Its own deployment pipeline
- Its own team who can do the following without input from others:
- Develop
- Test
- Deploy
- Make decisions
Structuring an application like this means:
- You can deliver updates much quicker as a component’s pipeline is shorter
- Updates shouldn’t delay or impact delivery to other components
Continuous Delivery recommends your development team structure mirrors your application’s architecture.
6: Database change management
Updating a database can be more disruptive or damaging than updating an application. Data changes quickly in modern software delivery and that can affect both deployments and your recovery options.
Say you remove a database column, row, or field between application versions. You could roll back to the previous version if the application update has major problems. A rollback becomes more difficult if the old version relies on the deleted information and users have already made changes.
That’s why it’s important to have strategies to help manage database changes during deployments. Your exact strategy depends on the type of application you develop.
There are a couple of common strategies to help manage database changes.
Have a database for each version
This strategy involves making your app’s database read-only during deployment and making a copy. You then upgrade and change the copy for the new version. Once happy, you redirect all traffic to the new app version.
It’s important to perform each step as quickly as possible. The longer you dwell, the higher the risk of recovery.
Decouple your database changes
When there’s one database for all versions, you can use an ‘expand and contract’ pattern to limit upgrade problems.
With this method, you can add to the database without impacting your application. The downside, however, is you can’t remove anything unless no running version depends on that information.
The pattern itself is to repeat 2 simple steps:
- Make a small change to the database
- Deploy
Let’s look at an example. If you wanted to change the ‘surname’ column’s label to ‘lastname’, you would:
- Add a new ‘lastname’ column to the database, and use a synchronization trigger so ‘lastname’ updates whenever ‘surname’ changes
- Deploy!
- Copy data from ‘surname’ to ‘lastname’
- Deploy!
- Update the application to use ‘lastname’ and remove all references to ‘surname’
- Deploy!
- Delete the synchronization trigger and ‘surname’ column, to prevent mistakes
- Deploy!
7: Deployment automation
Deployment automation helps you deploy more often and reliably, with the same packages and processes throughout a version’s lifecycle.
By automating your deployments, you achieve reliability through repetition. There should be no surprises deploying to Production if you already successfully deployed to Dev and Test.
Automating your deployment processes also helps you avoid human error. You’ll never miss steps or do them out of order. Plus, different teams can trigger deployments and not worry about making mistakes.
Deployment pipelines
A deployment pipeline is the mechanism to get a developer’s commits to environments.
A deployment pipeline uses the following tooling:
- Build server or CI platform
- Packaging service
- Deployment software
If you want to learn more about deployment pipelines read What is a deployment pipeline.
Why Continuous Delivery has become widely used
Continuous Delivery is popular because it’s statistically proven to work. You can measure how well your company uses Continuous Delivery with established frameworks and metrics. Use the following resources for guidance:
- Accelerate State of DevOps Report - An annual report that surveys companies using DevOps and shares the findings
- DORA (DevOps Research and Assessment) Metrics - A predictive way to measure DevOps practices and its outcomes
- The SPACE framework - A framework for managing the human part of your processes
For more information, read our Measuring Continuous Delivery whitepaper.
Help us continuously improve
Please let us know if you have any feedback about this page.
