In Octopus Deploy 3.0 we shipped first-class support for deploying Azure Web Apps. Rather than publishing directly from Visual Studio, you can manage environment-specific configuration and promote your application releases through environments in a controlled fashion.
We have been working with several customers where deploying Azure Web Apps were becoming unreliable. This is especially true when deploying large web apps to Azure, or when deploying over a slow or lossy network connection. In these circumstances deployments into an existing Azure Web App could become unreliable, hanging in certain cases, often requiring several attempts to deploy successfully. I hope it goes without saying - this is not the kind of experience we want for anyone using Octopus Deploy - we are all about Happy Deployments!
If you are having trouble with unreliable, slow, or hanging deployments for Azure Web Apps, this post is for you. In summary:
- Try timestamp instead of checksum
- Try deploying to a new slot and switching
- The future of Azure Web App deployments
A little explore under the covers...
When you deploy an Azure Web App with Octopus, we apply the usual Octopus goodness to your configuration files, run your custom scripts, and then rely on WebDeploy to sync the resulting files to Azure. We use the Microsoft.Web.Deployment NuGet package directly in code instead of shelling out to MsDeploy.exe.
Timestamp instead of checksum
In Octopus 3.0 we decided to make Checksum the default (and only choice) for syncing files with WebDeploy. Using checksum we would only deploy files which had actually changed, without needing any special configuration. This reduces the time and bandwidth required for subsequent deployments to the same web app. This works really nicely for moderate sized applications but, as we discovered, becomes prohibitively slow and unreliable for large applications.
If you find yourself in this situation you can switch to using Timestamp for file comparison instead of Checksum using Octopus Deploy 3.3.3 or newer.
If you have any existing Azure Web App steps you will need to update them to use timestamp and create a new release. Any new Azure Web App steps you create will use timestamp by default, but you can always switch to checksum if that suits your needs better.
If you prefer checksum, we are planning to upgrade to Microsoft.Web.Deployment.3.6.0 which has several bug fixes for checksum comparisons.
Preserving timestamps
The downside with using timestamps for file comparison is reliability - you can easily get false-positives - that's why checksum is so attractive. If you are using NuGet packages the timestamp of your files will be lost when they are unpacked meaning every file appears to be newer and will be uploaded regardless of whether it has changed - more detail. To work around this, consider using zip packages - we have automated tests proving the timestamps are preserved end-to-end through your deployments.
Try deploying to a new slot and switching
We often refer to these as blue-green deployments, where you deploy a new copy of your web app, run smoke tests, switch slots, and finally delete the old slot. Deploying over the top of an existing slot is convenient but cannot offer the same downtime as blue-green deployments.
We have written a short guide for using deployment slots with Azure Web Apps. For the cost of implementing one extra step in your deployment process this is a really good pattern for keeping your web apps up during deployment.
When you deploy to a brand new slot WebDeploy will just upload everything, there's really no file comparison to do, making large Web App deployments more reliable.
The future of Azure Web App deployments
We want to make sure you can deploy your ASP.NET Core 1 apps into the Azure App Service as soon as the final release is available. We already support JSON configuration files, and a build and deploy pipeline using zip packages.
Still to come:
- We are planning to upgrade to
Microsoft.Web.Deployment.3.6.0(beta release notes) - follow this GitHub Issue if you are interested in:- Deploying to folders above
wwwroot - Proxy support
- Bug fixes for sync using checksum
- Deploying to folders above
- We will implement deeper support for JSON configuration files
Still having problems?
If you are still experiencing problems with unreliable deployments into the Azure App Service, please get in touch with our support team and we will work with you to solve the specific problems you are experiencing.
