A growing trend among continuous integration and continuous delivery platforms is providing the ability to define pipelines as code, usually with YAML. One of the leaders in this area is CircleCI. In this post, we will look at a CircleCI configuration, including how to use and author CircleCI orbs. An orb is a reusable chunk of YAML that can be used across your CircleCI pipelines. Instead of copying and pasting the same code across multiple files, you can reference the functionality from the orb and keep your pipeline DRY.
What is CircleCI
CircleCI is a continuous integration platform that uses YAML based configurations to execute jobs on Docker containers or virtual machines. These jobs can be woven together into one or more workflows that will execute when you commit your code to source control.
Below is an example CircleCI job named build:
jobs:
build:
docker:
- image: mcr.microsoft.com/dotnet/core/sdk:2.1.607-stretch
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
steps:
- checkout
- run:
name: Install Cake
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet tool install -g Cake.Tool --version 0.35.0
- run:
name: Build
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet cake build.cake --target="Publish" --packageVersion="$PACKAGE_VERSION"
- persist_to_workspace:
root: publish
paths:
- "*"
Let's break this down piece by piece.
First, we define which docker image to execute our job on:
docker:
- image: mcr.microsoft.com/dotnet/core/sdk:2.1.607-stretch
We define an environment variable for our package version that includes the CircleCI pipeline number:
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
We define our steps. The first step checks out our source code:
steps:
- checkout
The second step installs Cake on the container executing the job:
- run:
name: Install Cake
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet tool install -g Cake.Tool --version 0.35.0
The third step invokes cake to build our project:
- run:
name: Build
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet cake build.cake --target="Publish" --packageVersion="$PACKAGE_VERSION"
The last step persists our published applications in the publish folder to a CircleCI workspace. Workspaces are used to share assets across multiple jobs:
- persist_to_workspace:
root: publish
paths:
- "*"
Let's take a look at a workflow that uses this job. When defining the workflow's jobs, we can set a job to depend on one or more other jobs. We've created a workflow called build that will run the jobs build, package, push, create-release, and deploy-release in that order:
workflows:
version: 2
build:
jobs:
- build
- package:
requires:
- build
- push:
requires:
- package
- create-release:
requires:
- push
- deploy-release:
requires:
- create-release
Reusable YAML and CircleCI orbs
Let's say that we have multiple projects that use Cake to build, and the steps are pretty much the same. Wouldn't it be great for each project to use a standard process without defining the same YAML in each configuration?
That's where orbs come in! Orbs define reusable executors (the environment your job will run in), commands that can be used in jobs, and jobs that can be used in workflows.
Using an orb
To use an orb, you import it with an orbs section in your configuration. In the section below, we've added the Slack orb by giving it the name slack and mapping it to the orb circleci/slack@3.4.2. circleci is the namespace that contains the orb. slack is the name of the orb. 3.4.2 is the version of the orb we want to use.
In our build job, we've added a new step - slack\notify. This step is the notify command from that orb. You can view the source of the command. That's quite a bit of YAML that we didn't need to include in our configuration.
version: 2.1
orbs:
slack: circleci/slack@3.4.2
jobs:
build:
docker:
- image: mcr.microsoft.com/dotnet/core/sdk:2.1.607-stretch
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
steps:
- checkout
- run:
name: Install Cake
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet tool install -g Cake.Tool --version 0.35.0
- run:
name: Build
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet cake build.cake --target="Publish" --packageVersion="$PACKAGE_VERSION"
- persist_to_workspace:
root: publish
paths:
- OctopusSamples.OctoPetShop.Database
- OctopusSamples.OctoPetShop.Infrastructure
- OctopusSamples.OctoPetShop.ProductService
- OctopusSamples.OctoPetShop.ShoppingCartService
- OctopusSamples.OctoPetShop.Web
- slack/notify:
message: Build ${PACKAGE_VERSION} succeeded
So let's create an orb for our Cake steps.
Creating an inline orb
In the example above, we imported an orb into our configuration. We can also define an orb directly in our configuration:
orbs:
cake:
executors:
cake-executor:
docker:
- image: mcr.microsoft.com/dotnet/core/sdk:2.1.607-stretch
jobs:
build:
executor: cake-executor
parameters:
cake_version:
default: "0.35.0"
type: string
package_version:
default: "$PACKAGE_VERSION"
type: string
publish_path:
default: "publish"
type: string
target:
default: "Publish"
type: string
steps:
- checkout
- run:
name: Install Cake
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet tool install -g Cake.Tool --version << parameters.cake_version >>
- run:
name: Build
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet cake build.cake --target="<< parameters.target >>" --packageVersion="<< parameters.package_version >>"
- persist_to_workspace:
root: << parameters.publish_path >>
paths:
- "*"
In the orb above, we've created an executor that defines the Docker image to use. We also defined a build job that accepts the Cake version, package version, publish path, and Cake target as parameters. It then uses those parameters in steps similar to those we had before.
With the orb defined, we can update our workflow to the following:
workflows:
version: 2
build:
jobs:
- cake/build:
package_version: 1.3.<< pipeline.number >>
- package:
requires:
- cake/build
- push:
requires:
- package
- create-release:
requires:
- push
- deploy-release:
requires:
- create-release
Now our workflow will use the build job from the Cake orb we defined in our configuration.
Publishing an orb
To use this orb in multiple projects, we need to publish it. To do that, we need to install and configure the CircleCI CLI.
After that's done, we create a namespace for our organization:
circleci namespace create octopus-samples github OctopusSamples # replace with your values
Then we create the orb in our namespace:
circleci orb create octopus-samples/cake # replace with your values
Now to get that inline orb published to CircleCI. We need to move our orb definition from our configuration into its own YAML file. We'll name it orb.yml.
version: 2.1
description: |
Orb for using Cake with OctopusSamples.
executors:
cake-executor:
docker:
- image: mcr.microsoft.com/dotnet/core/sdk:2.1.607-stretch
jobs:
build:
executor: cake-executor
parameters:
cake_version:
default: "0.35.0"
type: string
package_version:
default: "$PACKAGE_VERSION"
type: string
publish_path:
default: "publish"
type: string
target:
default: "Publish"
type: string
steps:
- checkout
- run:
name: Install Cake
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet tool install -g Cake.Tool --version << parameters.cake_version >>
- run:
name: Build
command: |
export PATH="$PATH:$HOME/.dotnet/tools"
dotnet cake build.cake --target="<< parameters.target >>" --packageVersion="<< parameters.package_version >>"
- persist_to_workspace:
root: << parameters.publish_path >>
paths:
- "*"
And then we can publish it with:
circleci orb publish orb.yml octopus-samples/cake@0.0.1
And presto! We have a published orb with our Cake build job. Now we can update our project's configuration to replace the inline orb with the published one:
orbs:
cake: octopus-samples/cake@0.0.1
Experimental Octopus CLI orb
Speaking of published orbs, if you're using CircleCI and Octopus together, take a peek at our experimental Octopus CLI orb. With this orb, you can use a subset of commands from the Octopus CLI to manage your packages, releases, and deployments:
orbs:
octo: octopusdeploylabs/octopus-cli@0.0.2
jobs:
package:
docker:
- image: ubuntu:18.04
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
steps:
- octo/install-tools
- attach_workspace:
at: publish
- octo/pack:
id: "OctopusSamples.OctoPetShop.Database"
version: "$PACKAGE_VERSION"
base_path: "publish/OctopusSamples.OctoPetShop.Database"
out_folder: "package"
- persist_to_workspace:
root: package
paths:
- OctopusSamples*
push:
docker:
- image: ubuntu:18.04
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
steps:
- octo/install-tools
- attach_workspace:
at: package
- octo/push:
package: "package/OctopusSamples.OctoPetShop.Database.${PACKAGE_VERSION}.zip"
server: "$OCTOPUS_SERVER"
api_key: "$OCTOPUS_API_KEY"
debug: true
- octo/build-information:
package_id: "OctopusSamples.OctoPetShop.Database"
version: "$PACKAGE_VERSION"
server: "$OCTOPUS_SERVER"
api_key: "$OCTOPUS_API_KEY"
debug: true
create-release:
docker:
- image: ubuntu:18.04
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
steps:
- octo/install-tools
- octo/create-release:
project: "Octo Pet Shop"
server: "$OCTOPUS_SERVER"
api_key: "$OCTOPUS_API_KEY"
release_number: $PACKAGE_VERSION
As a bonus, we can continue using an inline orb to reduce our duplication even more. Let's make a wrapper around octo-exp\pack:
orbs:
cake: octopus-samples/cake@0.0.1
octo: octopusdeploylabs/octopus-cli@0.0.2
octopetshop:
orbs:
octo: octopusdeploylabs/octopus-cli@0.0.2
commands:
pack:
parameters:
id:
type: string
steps:
- octo/pack:
id: "<< parameters.id >>"
version: $PACKAGE_VERSION
base_path: "publish/<< parameters.id >>"
out_folder: "package"
jobs:
package:
executor:
- name: octo/default
environment:
PACKAGE_VERSION: 1.3.<< pipeline.number >>
steps:
- octo/install-tools
- attach_workspace:
at: publish
- octopetshop/pack:
id: "OctopusSamples.OctoPetShop.Database"
- octopetshop/pack:
id: "OctopusSamples.OctoPetShop.Infrastructure"
- octopetshop/pack:
id: "OctopusSamples.OctoPetShop.ProductService"
- octopetshop/pack:
id: "OctopusSamples.OctoPetShop.ShoppingCartService"
- octopetshop/pack:
id: "OctopusSamples.OctoPetShop.Web"
- persist_to_workspace:
root: package
paths:
- OctopusSamples*
Since our calls to octo/pack will follow the same format with different values for id, we can use octopetshop\pack to abstract that logic away.
Conclusion
CircleCI Orbs are a way to create reusable commands or jobs for your YAML based pipelines. You can publish orbs to CircleCI to share across projects or with other organizations. You can also use inline orbs to aid in development or to cut down the noise in your own configurations.
Check out CircleCI Orbs for more information and the experimental Octopus CLI orb for details on how you can use CircleCI and Octopus together.
