and Azure DevOps
You may already use Azure DevOps, or you might be planning your next CI/CD pipeline. Octopus Deploy brings best-in-class deployments, release management, and operations runbooks capabilities, giving your team power far beyond what's included with Azure DevOps. This page explains the difference between the two, and why the best engineering teams use them together.
Developers often ask us why they should use Octopus Deploy over Azure DevOps, especially if they already use Azure DevOps for bug tracking, builds, or source control. This page is our attempt to explain the major differences in capabilities, and why you'd want to use both together.
Obviously, we're biased — we really like Octopus Deploy! But we do want this page to be fair and accurate. If you see anything here that you disagree with, let us know and we'll correct it quickly.
About Azure DevOps
Azure DevOps is Microsoft's end-to-end solution for development teams. It provides numerous features, including source code repository, requirements management, project management, automated builds, lab management, as well as release management.
The Azure DevOps platform started life as Team Foundation Server (TFS) in 2005 and has gone through a number of renames since, including Visual Studio Online and Visual Studio Team System/Services (VSTS).
When Octopus is compared to Azure DevOps, it's really about comparing Octopus Deploy to the Release Management features of Azure DevOps, as Octopus doesn't attempt to do bug tracking, source control, or builds.
The release management story in Azure DevOps began when Microsoft acquired InRelease in 2013. Most TFS users found InRelease limited and confusing, and they pushed Microsoft to look at other solutions, including Octopus Deploy, as examples of how release management could be done better. In 2015, Microsoft rebuilt the release management aspects of TFS (then VSTS at the time) on the same core as their new build platform.
In 2018, Microsoft acquired GitHub, and recently Microsoft's focus seems to be moving to GitHub Actions, with a number of key people involved in Azure DevOps moving over to GitHub. While we expect Azure DevOps to be supported for a long time, it's not clear how the roadmap will evolve or how much Microsoft will continue to invest in it.
About Octopus Deploy
Octopus Deploy is a best-in-class platform for release management, deployment automation, and operations runbook automation.
Unlike Azure DevOps, Octopus doesn't provide bug tracking, build automation, or source control. We're focused on building software that does a few things very well and integrating well with other best-in-class tooling.
Octopus Deploy is an independent software vendor. It was founded in 2011 and is bootstrapped, conservatively managed, and profitable. Octopus Deploy's goal is to build the most powerful and capable solution for release management, deployment, and operations automation. Learn more about our company.
Highlights & key differences
While there are many differences between Octopus Deploy and the pipelines/release management aspects of Azure DevOps, the table below summarizes the key differences.
|Azure DevOps||Octopus Deploy|
|Bug tracking, source control, build/CI automation Octopus Deploy does not attempt to do any of these, as teams already have many solutions to pick from. You can use Azure DevOps for these features, and then use Octopus for release management, deployments, and operations. Alternatively, you can combine other best-in-class solutions with Octopus for the ultimate CI/CD pipeline.|
|Release management Both Azure DevOps and Octopus Deploy can model environments like dev, staging, and production, and both can model approvals and permissions around who can deploy to each.|
|Variable management Variables can store API keys, connection strings, file paths, passwords, and other settings that vary during a deployment. Both Octopus and Azure DevOps release pipelines can model variables. Variables in Azure DevOps can only be scoped by environments. Variables in Octopus are much more powerful - they can be scoped to individual targets, steps, and can be shared between projects.|
DevOps teams don't just need to automate deployments, they also need to automate a variety of emergency and routine tasks. This might mean
updating certificates, disaster recovery, or ad-hoc management tasks. Runbooks makes this possible in Octopus.
Octopus Deploy is the only release management tool that can also be used by operations teams, giving teams a unified platform for DevOps automation. Despite the name, Azure DevOps doesn't actually automate the ops side of DevOps; teams need to find another solution.
|Multi-tenancy If you deploy a copy of your software for every customer that signs up, you might find yourself with dozens, hundreds, or even thousands of deployment pipelines. Octopus is the only release automation server with first-class multi-tenancy support, letting you deploy to a single customer or a group without duplication. With Azure DevOps, you need to duplicate your release definitions for every customer.|
Configuration as code
Azure DevOps has YAML support for Pipelines, but this is primarily builds. The Releases feature in Azure DevOps
does not support YAML, and it does not appear to be on Microsoft's roadmap.
Multi-stage pipelines can be used in scenarios where "every commit goes to production", but it loses all the release management capabilities of Azure DevOps.
We are working on Configuration as Code support in Octopus Deploy, which will use the HCL language and store your deployment process and variables in Git. A key difference, however, is that the UI will still work in Octopus. Many developers who start using YAML in Azure DevOps find it both a blessing and a curse - they want the source control aspects, but writing YAML is difficult, and it's hard for the team to understand how the deployment works. We think our approach brings the best of both worlds.
|Not for release management.|
|Cloud/PaaS support Azure DevOps natively supports Azure, with marketplace extensions available for AWS and Google Cloud. Octopus natively supports Azure and AWS, with support for Kubernetes in Google Cloud, and full GCP support coming soon.|
|On-premises, cloud VM/IaaS support. If you run your own VM's in the cloud or on premises, there aren't a lot of options native to Azure DevOps. The supported IIS steps require WinRM/WebDeploy, which limits the types of networks (e.g., same AD domain) you can authenticate on. Octopus Deploy has the most powerful on-premises support, with a smart agent called the Tentacle, and a lot of networking options.|
YAML vs Classic pipelines
Azure DevOps implements two distinct pipeline styles: classic and YAML. The Microsoft documentation provides a comparison of the features available for each style.
Classic and YAML pipelines have differences in the steps they can use, the security boundaries they expose, the concepts they use to model deployments, and the high level workflows they facilitate. Teams looking to automate their deployments in Azure DevOps must make the decision early on as to which pipeline style to use, or support both styles.
Microsoft has been careful not to suggest that one pipeline style is recommended or preferred over the other. However, you can find hints in issues, release notes, and public forums that indicate Microsoft's commitment to each pipeline style.
For example, the GitHub issue What's the long term plan for Classic Release Pipelines? has this response:
Currently there are no plans to deprecate Classic pipelines. However, new features are being planned for YAML first.
This sentiment is reflected in the release notes for Azure DevOps 2022, which lists one "feature" for classic builds (Removal of per-pipeline retention policies in classic builds), and six features for YAML pipelines.
Octopus provides a single, unified deployment experience regardless of whether you wish to define your deployment process as code or not, so there is no need to support two distinct styles.
Limitations of orchestrating Continuous Delivery with YAML pipelines
With the bias towards YAML pipelines in mind, it is worth understanding the limitations of multi-stage YAML pipelines when performing deployments.
Continuous Delivery is a largely automated deployment process that includes manual touch points. For example, release managers may be required to initiate a production deployment. Continuous Deployment, on the other hand, is a completely automated workflow where every commit is deployed to production with no manual intervention, so long as all the tests and other automated requirements pass.
While Continuous Deployments have many benefits, most teams we talk to find Continuous Delivery works for them.
Because of the way they were designed, Azure DevOps multi-stage YAML pipelines don't provide a great Continuous Delivery experience. Each execution of a YAML pipeline expects to run to completion, which in a typical implementation is to complete a production deployment, unless a ManualValidation step or Environment approval times out or is rejected. Rejected approvals result in a failed pipeline, despite the fact that most builds are not expected to be deployed to production when practising Continuous Delivery.
The GitHub issue How to just do manual propagation instead of approvals? discusses this issue, with comments like this highlighting the fundamental problem:
I also need the ability to deploy manually to environments. Setting approvers is insufficient because not every build will go to production. This means most builds would be considered failed due to a timeout, which defeats the entire purpose of the "pipeline failure" metric.
Octopus fully supports both Continuous Delivery and Continuous Deployment. Promoting a release to the next environment is as easy as clicking the DEPLOY button, without any implicit requirement that releases always make it to production. Or, if automated deployments to the production environment are required, deployments are automatically triggered on the next environment when a previous deployment completes successfully.
Classic pipelines lack global environment security controls
Continuous Delivery often requires the ability to restrict production deployments to a small set of trusted individuals. This means implementing security rules that:
- Can define what the production environment is.
- Can enforce rules that allow select individuals to perform deployments to the production environment.
Classic releases in Azure DevOps define environments as stages. However, stages are release pipeline specific entities and each have their own security policies. This means that even if two release pipelines define a stage called "Production", both stages are unique entities, and both must be secured individually.
Deployment groups are a second resource used to define environments. However, there are no permissions or approvals that can be applied to deployment groups to control who initiates a deployment.
What this means in practice is that release pipelines must either:
- Individually define security rules on their production stage.
- Individually define approval gates before a production deployment is initiated.
It is easy to see how this process becomes unwieldy when you have dozens or hundreds of release pipelines. Assuming each release pipeline clearly defines a stage representing a production deployment (which is not guaranteed) security teams still need to individually implement or verify the permissions assigned to the production stage or the approval gates.
The Stack Overflow question How [do I] prevent users from deploying to a Deployment group discusses this limitation.
Octopus environments are global resources used to group infrastructure such as accounts and targets. Environments are also placed in lifecycle phases determining the order in which releases are deployed. Critically, environments also provide a security boundary that cross cutting permissions are applied to.
There is no question as to what your production environment is in Octopus: it is literally the environment called "Production". Security rules are applied to environments at a global level providing certainty that your deployments are secure.
Runbooks: automation for Ops too
Modern operations teams often need to automate a huge variety of tasks:
Routine operations, especially for tasks that don't happen very frequently. For example:
- Backing up and testing the restore of databases.
- Requesting and updating SSL certificates.
- Resetting API keys or rotating passwords.
- Provisioning or deprovisioning infrastructure.
- Suspending or taking applications offline for maintenance.
- Updating a test environment with fresh, sanitized production data.
Emergency operations, such as tasks you might need to perform at 3 AM
after an alert is raised.
- Failing over to a disaster recovery site, and then back to the main site.
- Restarting a set of applications, perhaps in a specific order.
- Restarting a web application that experiences a periodic memory leak that is yet to be resolved.
Whether you use Octopus or Azure DevOps for releases, you'll likely need to tell your release management solution a whole lot of things:
- Access to your Azure/AWS subscriptions.
- The various servers, cloud targets, and other resources your software runs on.
- Certificates and API keys for various other platforms you use.
- Database and other credentials for schema updates.
- Permissions and controls around who can deploy.
If you're going to set all of this up for deployments, you should be able to use it for other automation tasks too.
In addition to automating deployments, Octopus also provides first-class support for Runbooks, which can be used to automate these routine and emergency operations tasks. Runbooks in Octopus belong to the same project as your deployment process, and they have access to the same environments, variables, certificates, and everything else. It's the closest thing to a complete DevOps automation platform.
Despite the name, Azure DevOps has no ability to automate operations tasks that are not deployments. Microsoft's recommendation is to use Azure Automation, another solution. Nothing is synchronized between them, so you'll end up duplicating all of this configuration, with no source of truth. The silo between Dev and Ops will remain, each using their own automation solution.
Configuration as code
Octopus and Azure DevOps Releases use a UI to define the individual steps for deployments. Most developers on a team aren't experts in build/deployment automation, and co-ordinating complex deployments across many machines doesn't lend itself to a scripting language. The UI makes it easy for people - especially those who didn't author the original deployment - to figure out what is going on.
Starting in 2018, Azure DevOps Pipelines added support for writing build/CI pipelines in YAML. In fact, for builds, the UI-driven way of automating builds has been renamed "Classic Pipelines", a signal that YAML is the future.
Whether you like the choice of YAML or not, you'll probably agree Configuration as Code is a good idea. So you might be surprised to learn that Releases in Azure DevOps still use the "Classic" UI-driven solution - there's no YAML support for releases. Microsoft originally indicated it would happen, but so far it hasn't, and it's not on their public roadmap.
It seems as though Releases (and Release Management) in Azure DevOps has been ignored for YAML, in favor of "multi-stage Pipelines" which use YAML. However, multi-stage pipelines assume that the entire pipeline will execute (unless there's a failure) all the way, for every commit. This makes sense if you want every commit to go to production as soon as it's built, but not if you have any kind of approval process (e.g., a QA team), or designated times when deployment is safe.
The summary is, in Azure DevOps, you can use YAML, or you can use release management, but not both. And there's nothing on the roadmap suggesting Release Management will get YAML support.
With Config as Code in Octopus, we provide the full power of Git with the usability of Octopus. This means you can see your deployment process in your Git repo next to your application code and evolve them together. When Config as Code is enabled for a project, you can continue to use the Octopus UI as you always have, or you can edit the text files in your favorite editor.
Our Config as Code solution is different. Importantly, the Octopus UI still works, so you don’t need to learn a whole new language to edit your process. We want you to have a great experience using the Octopus Web Portal so you can configure your Git repository, select your branch, create new branches, commit changes, and more, all from the Octopus UI.
We believe branches are Git's superpower, and we wanted to expose that power. You can switch and create branches, in the Octopus UI, allowing you to make changes to your deployment process on a branch. This effectively allows for a draft mode, as you can then create a release from your branch and test your changes. When combined with features from your Git provider, like pull requests, protected branches, and code-owners, this enables a whole new set of workflows.
Our configuration file format is Octopus Configuration Language (OCL), based on HashiCorp's HCL. We wanted our configuration files to be human readable and to support complex documents like deployment and runbook processes. We like HCL and feel it's the right tool for the job, but we've written our own parser/serializer. This means we're not obligated to follow any direction HashiCorp takes HCL, and there's nothing preventing us from making changes.
If you prefer full control, you can edit the underlying configuration files but this isn't required. The rich two-way sync means you can mix and match your use of the Octopus UI and direct editing. We also built a Visual Studio Code extension so you can edit OCL files with syntax highlighting, code snippets and fast navigation.
If you deploy a copy of your software for every customer that signs up, you might find yourself with dozens, hundreds, or even thousands of deployment pipelines.
Octopus is the only release management & deployment automation solution with first-class multi-tenancy support, letting you manage deployments to a single customer or a group without duplication. With Azure DevOps, you need to duplicate your release definitions for every customer.
Multi-tenancy is particularly suited to scenarios like:
- White-labeled software.
- SaaS software where every customer gets their own database.
- Managed services.
- Head-office deployments to restaurants, retail stores, and more.
“Octopus Deploy is the most sophisticated release management solution. It is the broadest, and the deepest deployment tool for any kind of application on the Microsoft platform.”Jeffrey Palermo, Microsoft MVP & Chief Architect of Clear Measure
Host of the Azure DevOps Podcast
Use Octopus Deploy with Azure DevOps
Octopus with Azure DevOps can be a very powerful combination. With Azure DevOps, you get source control, build/CI, and bug tracking. With Octopus, you get the most powerful release management, deployment and runbook automation capabilities. See the diagram below for an overview of how they work together: