Managing AWS costs with Instance Scheduler

The promise of cloud computing has been to allow teams to efficiently scale up and down on demand. While on-premises infrastructure can save on electricity and cooling costs by shutting down, cloud-based resources can avoid almost all charges (storage fees usually apply to stopped resources) by stopping any that are unused.

AWS provides the Instance Scheduler to shutdown and restart EC2 and RDS resources on demand. This is a great solution for teams that have Octopus resources like Workers running in AWS, and where they are unused for most of the day.

In this post, you learn how to install the Instance Scheduler, configure it with custom periods, and tag resources to be automatically shutdown and restarted.

Prerequisites

This post assumes you'll be running scripts on a Linux Worker. You need Python 3, jq, curl, and unzip installed to complete the steps in this post. To install these tools in Ubuntu, run the command:

apt-get install jq curl unzip python3

To install the tools in Fedora, RHEL, Centos, and Amazon Linux, run the command:

yum install jq curl unzip python3

The Instance Scheduler is managed with a custom Python application called Scheduler CLI.

The CLI requires Python 3. If you have Python 2 and Python 3 installed, you can force the use of Python 3 with the command:

alias python=python3

Run the following commands to install the CLI:

curl -O https://s3.amazonaws.com/solutions-reference/aws-instance-scheduler/latest/scheduler-cli.zip
unzip scheduler-cli.zip
python setup.py install

A public Octopus instance has been configured with a project that deploys the Instance Scheduler.

Deploying the Instance Scheduler template

Instance Scheduler is distributed as a CloudFormation template for download. You'll deploy this with the Deploy an AWS CloudFormation template step in Octopus.

You can leave the parameters with their default values, although you likely want to define the DefaultTimezone parameter to reflect your local timezone.

We created a live example of this step.

Adding new periods

The Instance Scheduler works by defining a number of periods in a DynamoDB database. The Scheduler CLI provides a convenient interface through which these periods can be viewed and manipulated.

The script below calls scheduler-cli describe-periods to list the currently defined periods, and then pipes the resulting JSON to jq, which checks to see if a period called aus_weekday exists. If it does not exist, the period is added by calling scheduler-cli create-period. If it does exist the period is updated by calling scheduler-cli update-period.

This script is necessary to create an idempotent deployment, where a deployment can be redeployed at any point, regardless of the existing state of the database.

This script is run with the Run an AWS CLI Script step in Octopus. You can see this in our live example:

alias python=python3
scheduler-cli describe-periods --stack common-instance-scheduler | jq -e '.Periods|any(.Name == "aus_weekday")' > /dev/null
if [[ 0 -ne $? ]]; then
    scheduler-cli create-period --name "aus_weekday" --begintime 05:00 --endtime 17:00 --weekdays mon-fri --stack common-instance-scheduler
else
    scheduler-cli update-period --name "aus_weekday" --begintime 05:00 --endtime 17:00 --weekdays mon-fri --stack common-instance-scheduler
fi

Tagging resources

The Instance Scheduler identifies resources to shutdown and restart based on tags. The default tag the Instance Scheduler looks for is called Schedule (although the tag name can be modified by changing the TagName parameter in the CloudFormation template). The value of the tag is set to the name of a period.

In the screenshot below, you can see that a worker EC2 instance has a tag called Schedule set to aus_weekday. This means this EC2 instance will be shutdown at 17:00 and restarted at 05:00 every weekday. This more than halves the running costs of the EC2 instance as it's now shutdown overnight and over the weekend:

Conclusion

The AWS Instance Scheduler is a convenient solution that lets you automatically shutdown and restart EC2 and RDS instances, which can significantly reduce running costs for resources that aren't required 24 hours a day.

In this post, you saw how to deploy the Instance Scheduler with Octopus, add a new period with the Scheduler CLI, and tag your resources to allow the Scheduler to shut them down overnight and over the weekend.

Read the rest of our Runbooks series.

Happy deployments!