The promise of cloud computing has been to allow teams to efficiently scale up and down on demand. While on-premises infrastructure can save on electricity and cooling costs by shutting down, cloud-based resources can avoid almost all charges (storage fees usually apply to stopped resources) by stopping any that are unused.
AWS provides the Instance Scheduler to shutdown and restart EC2 and RDS resources on demand. This is a great solution for teams that have Octopus resources like Workers running in AWS, and where they are unused for most of the day.
In this post, you learn how to install the Instance Scheduler, configure it with custom periods, and tag resources to be automatically shutdown and restarted.
This post assumes you'll be running scripts on a Linux Worker. You need Python 3,
unzip installed to complete the steps in this post. To install these tools in Ubuntu, run the command:
apt-get install jq curl unzip python3
To install the tools in Fedora, RHEL, Centos, and Amazon Linux, run the command:
yum install jq curl unzip python3
The Instance Scheduler is managed with a custom Python application called Scheduler CLI.
The CLI requires Python 3. If you have Python 2 and Python 3 installed, you can force the use of Python 3 with the command:
Run the following commands to install the CLI:
curl -O https://s3.amazonaws.com/solutions-reference/aws-instance-scheduler/latest/scheduler-cli.zip unzip scheduler-cli.zip python setup.py install
Deploying the Instance Scheduler template
Instance Scheduler is distributed as a CloudFormation template for download. You'll deploy this with the Deploy an AWS CloudFormation template step in Octopus.
If you see the error
Template could not be parsed: An item with the same key has already been added. Key: 14 when attempting to save the template, it's because one of the parameters has duplicated an option in the
AllowedValues array. In the screenshot below, you can see the
LogRetentionDays parameter has duplicated value
14. To resolve the error, remove the duplicated value:
You can leave the parameters with their default values, although you likely want to define the
DefaultTimezone parameter to reflect your local timezone.
We created a live example of this step.
Adding new periods
The Instance Scheduler works by defining a number of periods in a DynamoDB database. The Scheduler CLI provides a convenient interface through which these periods can be viewed and manipulated.
The script below calls
scheduler-cli describe-periods to list the currently defined periods, and then pipes the resulting JSON to
jq, which checks to see if a period called
aus_weekday exists. If it does not exist, the period is added by calling
scheduler-cli create-period. If it does exist the period is updated by calling
This script is necessary to create an idempotent deployment, where a deployment can be redeployed at any point, regardless of the existing state of the database.
This script is run with the Run an AWS CLI Script step in Octopus. You can see this in our live example:
alias python=python3 scheduler-cli describe-periods --stack common-instance-scheduler | jq -e '.Periods|any(.Name == "aus_weekday")' > /dev/null if [[ 0 -ne $? ]]; then scheduler-cli create-period --name "aus_weekday" --begintime 05:00 --endtime 17:00 --weekdays mon-fri --stack common-instance-scheduler else scheduler-cli update-period --name "aus_weekday" --begintime 05:00 --endtime 17:00 --weekdays mon-fri --stack common-instance-scheduler fi
The Instance Scheduler identifies resources to shutdown and restart based on tags. The default tag the Instance Scheduler looks for is called
Schedule (although the tag name can be modified by changing the
TagName parameter in the CloudFormation template). The value of the tag is set to the name of a period.
In the screenshot below, you can see that a worker EC2 instance has a tag called
Schedule set to
aus_weekday. This means this EC2 instance will be shutdown at 17:00 and restarted at 05:00 every weekday. This more than halves the running costs of the EC2 instance as it's now shutdown overnight and over the weekend:
The AWS Instance Scheduler is a convenient solution that lets you automatically shutdown and restart EC2 and RDS instances, which can significantly reduce running costs for resources that aren't required 24 hours a day.
In this post, you saw how to deploy the Instance Scheduler with Octopus, add a new period with the Scheduler CLI, and tag your resources to allow the Scheduler to shut them down overnight and over the weekend.
Follow our series about Runbooks
We're focusing on Runbooks for a few months this year. We’ll include a summary of the blog posts in our monthly newsletter. Sign up to follow along.