AWS Secrets Manager - Retrieve Secrets

To use this template in Octopus Deploy, copy the JSON below and paste it into the Library → Step templates → Import dialog.

{
  "Id": "5d5bd3ae-09a0-41ac-9a45-42a96ee6206a",
  "Name": "AWS Secrets Manager - Retrieve Secrets",
  "Description": "This step retrieves one or more secrets from AWS [Secrets Manager](https://aws.amazon.com/secrets-manager) and creates [sensitive output variables](https://octopus.com/docs/projects/variables/output-variables#sensitive-output-variables) for each value retrieved. The step supports creating a variable for each key-value in a secret that's retrieved, or you can specify individual keys. These values can be used in other steps in your deployment or runbook process.\n\n---\n\n**Specifying Secret names/keys to retrieve:**\n\nSpecify the names of the secrets to be returned from AWS Secrets Manager, in the format:\n\n`SecretName SecretVersionId SecretVersionStage | KeyNames | OutputVariableName` where:\n\n- `SecretName` is the name of the secret to retrieve. You can specify either the `Amazon Resource Name (ARN)` or the friendly name of the secret.\n- `SecretVersionId` is the unique identifier of the version of the secret that you want to retrieve. If this value isn't specified, the version with the `VersionStage` value as specified in `SecretVersionStage` will be retrieved.\n- `SecretVersionStage` specifies the secret version that you want to retrieve by the staging label attached to the version. *Staging labels are used to keep track of different versions during the rotation process*. If this value isn't specified, the version with the `VersionStage` value of `AWSCURRENT` will be retrieved.\n- `KeyNames` are the names of the keys stored in the secret that you wish to retrieve values for. Multiple fields can be retrieved separated by a space. Alternatively, you can specify all fields using the special keyword `all` or `*`.\n- `OutputVariableName` is the _optional_ Octopus [output variable](https://octopus.com/docs/projects/variables/output-variables) name to store the secret's value in. If multiple fields are specified the field name will be appended to this value. *If this value isn't specified, an output name will be generated dynamically*.\n\n**Examples:**\n\nGiven a secret named `OctoSamples-usercredentials`:\n\n1. `OctoSamples-usercredentials | Username | octousername`\n   \n   This would retrieve the secret and extract the value from the key-value named `Username` and save it into a sensitive output variable named `octousername`.\n\n2. `OctoSamples-usercredentials | Username Password | octocreds`\n   \n   This would retrieve the secret named `OctoSamples-usercredentials`, and then extract the values from the key-values named `Username` and `Password` and save them to two sensitive output variables named `octocreds.Username` and `octocreds.Password`.\n\n3. `OctoSamples-usercredentials | * | octocreds`\n   \n   This would retrieve the secret named `OctoSamples-usercredentials`, and then extract all key-values from the secret and save them to sensitive output variables *prefixed* with `octocreds`.\n\n4. `OctoSamples-usercredentials | all`\n   \n   This would retrieve the secret named `OctoSamples-usercredentials`, and then extract all key-values from the secret and save them to sensitive output variables *prefixed* with `OctoSamples-usercredentials`.\n\n---\n\n**AWS Dependencies:**\n\nThere are some dependencies/requirements for this step to work successfully.\n\n1. **CLI** - This step uses AWS tooling pre-installed on the target or worker. \n\n    Scripts executed in this step need to use the [AWS CLI](https://aws.amazon.com/cli/) to authenticate to AWS and perform other actions. If the CLI can't be found, the step will fail.\n\n2. **AWS Account** - An [AWS account](https://octopus.com/docs/infrastructure/accounts/aws) with permissions to retrieve secrets from AWS Secrets Manager is also required.\n\n---\n\n**Notes:** \n- Tested on Octopus **2021.2**.\n- Tested on both Windows Server 2019 and Ubuntu 20.04.\n\n",
  "Version": 4,
  "ExportedAt": "2023-04-22T17:43:19.580Z",
  "ActionType": "Octopus.AwsRunScript",
  "Author": "harrisonmeister",
  "Packages": [],
  "Parameters": [
    {
      "Id": "8623cdbe-f962-4801-9470-5d14d1d7d5ed",
      "Name": "AWS.SecretsManager.RetrieveSecrets.Account",
      "Label": "AWS Account",
      "HelpText": "An AWS account with permissions to access secrets from Secrets Manager.",
      "DefaultValue": "",
      "DisplaySettings": {
        "Octopus.ControlType": "AmazonWebServicesAccount"
      }
    },
    {
      "Id": "55a1d3e8-90c8-4c1a-a315-246fd8660e81",
      "Name": "AWS.SecretsManager.RetrieveSecrets.Region",
      "Label": "AWS Region",
      "HelpText": "Specify the default region. View the [AWS Regions and Endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region) documentation for a current list of the available region codes.",
      "DefaultValue": "",
      "DisplaySettings": {
        "Octopus.ControlType": "SingleLineText"
      }
    },
    {
      "Id": "e204a03d-80a6-437a-9a8b-8812c299741c",
      "Name": "AWS.SecretsManager.RetrieveSecrets.SecretNames",
      "Label": "Secret names to retrieve",
      "HelpText": "Specify the names of the secrets to be returned from AWS Secrets Manager, in the format:\n\n`SecretName SecretVersionId SecretVersionStage | KeyNames | OutputVariableName` where:\n\n- `SecretName` is the name of the secret to retrieve. You can specify either the `Amazon Resource Name (ARN)` or the friendly name of the secret.\n- `SecretVersionId` is the unique identifier of the version of the secret that you want to retrieve. If this value isn't specified, the version with the `VersionStage` value as specified in `SecretVersionStage` will be retrieved.\n- `SecretVersionStage` specifies the secret version that you want to retrieve by the staging label attached to the version. *Staging labels are used to keep track of different versions during the rotation process*. If this value isn't specified, the version with the `VersionStage` value of `AWSCURRENT` will be retrieved.\n- `KeyNames` are the names of the keys stored in the secret that you wish to retrieve values for. Multiple fields can be retrieved separated by a space. Alternatively, you can specify all fields using the special keyword `all` or `*`. *See the step description for examples*.\n- `OutputVariableName` is the _optional_ Octopus [output variable](https://octopus.com/docs/projects/variables/output-variables) name to store the secret's value in. If multiple fields are specified the field name will be appended to this value. *If this value isn't specified, an output name will be generated dynamically*.\n\n**Note:** Multiple fields can be retrieved by entering each one on a new line.",
      "DefaultValue": "",
      "DisplaySettings": {
        "Octopus.ControlType": "MultiLineText"
      }
    },
    {
      "Id": "17ba53a4-bf94-498c-8905-0d37b86eaeea",
      "Name": "AWS.SecretsManager.RetrieveSecrets.PrintVariableNames",
      "Label": "Print output variable names",
      "HelpText": "Write out the Octopus [output variable](https://octopus.com/docs/projects/variables/output-variables) names to the task log. Default: `False`.",
      "DefaultValue": "False",
      "DisplaySettings": {
        "Octopus.ControlType": "Checkbox"
      }
    }
  ],
  "Properties": {
    "Octopus.Action.Script.ScriptSource": "Inline",
    "Octopus.Action.Script.Syntax": "PowerShell",
    "Octopus.Action.Aws.AssumeRole": "False",
    "Octopus.Action.AwsAccount.UseInstanceRole": "False",
    "OctopusUseBundledTooling": "False",
    "Octopus.Action.Script.ScriptBody": "$ErrorActionPreference = 'Stop'\n\n# Variables\n$SecretNames = $OctopusParameters[\"AWS.SecretsManager.RetrieveSecrets.SecretNames\"]\n$PrintVariableNames = $OctopusParameters[\"AWS.SecretsManager.RetrieveSecrets.PrintVariableNames\"]\n\n# Validation\nif ([string]::IsNullOrWhiteSpace($SecretNames)) {\n    throw \"Required parameter AWS.SecretsManager.RetrieveSecrets.SecretNames not specified\"\n}\n\n# Functions\nfunction Format-SecretName {\n    [CmdletBinding()]\n    Param(\n        [string] $Name,\n        [string] $VersionId,\n        [string] $VersionStage,\n        [string[]] $Keys\n    )\n    $displayName = \"'$Name'\"\n    if (![string]::IsNullOrWhiteSpace($VersionId)) {\n        $displayName += \" $VersionId\"\n    }\n    if (![string]::IsNullOrWhiteSpace($VersionStage)) {\n        $displayName += \" $VersionStage\"\n    }\n    if ($Keys.Count -gt 0) {\n        $displayName += \" ($($Keys -Join \",\"))\"\n    }\n    return $displayName\n}\n\nfunction Save-OctopusVariable {\n    Param(\n        [string] $name,\n        [string] $value\n    )\n    if ($script:storedVariables -icontains $name) {\n        Write-Warning \"A variable with name '$name' has already been created. Check your secret name parameters as this will likely cause unexpected behavior and should be investigated.\"\n    }\n    Set-OctopusVariable -Name $name -Value $value -Sensitive\n    $script:storedVariables += $name\n\n    if ($PrintVariableNames -eq $True) {\n        Write-Host \"Created output variable: ##{Octopus.Action[$StepName].Output.$name}\"\n    }\n}\n\n# End Functions\n\n$script:storedVariables = @()\n$StepName = $OctopusParameters[\"Octopus.Step.Name\"]\n$Secrets = @()\n\n# Extract secret names\n@(($SecretNames -Split \"`n\").Trim()) | ForEach-Object {\n    if (![string]::IsNullOrWhiteSpace($_)) {\n        Write-Verbose \"Working establishing secret definition for: '$_'\"\n        $secretDefinition = ($_ -Split \"\\|\")\n        \n        # Establish the secret name/version requirements\n        $secretName = $secretDefinition[0].Trim()\n        $secretVersionId = \"\"\n        $secretVersionStage = \"\"\n        $secretNameAndVersion = ($secretName -Split \" \")\n        \n        if ($secretNameAndVersion.Count -gt 1) {\n            $secretName = $secretNameAndVersion[0].Trim()\n            $secretVersionId = $secretNameAndVersion[1].Trim()\n            if ($secretNameAndVersion.Count -eq 3) {\n                $secretVersionStage = $secretNameAndVersion[2].Trim()\n            }\n        }\n    \n        if ([string]::IsNullOrWhiteSpace($secretName)) {\n            throw \"Unable to establish secret name from: '$($_)'\"\n        }\n\n        # Establish the secret field(s)/output variable name requirements.\n        $VariableName = \"\"\n        $Keys = @()\n        if ($secretDefinition.Count -gt 1) {\n            $KeyNames = $secretDefinition[1].Trim()        \n            $Keys = @(($KeyNames -Split \" \"))\n            $EmptyKeys = $Keys | Where-Object { [string]::IsNullOrWhiteSpace($_) }\n            if ($Keys.Count -le 0 -or $EmptyKeys.Count -gt 0) {\n                throw \"No keys (field names) were specified for '$_'. To retrieve all keys in a secret, add the word ALL or the wildcard (*) character.\"    \n            }\n            \n            if ($secretDefinition.Count -gt 2) {\n                $VariableName = $secretDefinition[2].Trim()\n            }\n        }\n        else {\n            throw \"No keys (field names) were specified for '$_'. To retrieve all keys in a secret, add the word ALL or the wildcard (*) character.\"\n        }\n\n        $secret = [PsCustomObject]@{\n            Name                 = $secretName\n            SecretVersionId      = $secretVersionId\n            SecretVersionStage   = $secretVersionStage\n            Keys                 = $Keys\n            variableNameOrPrefix = $VariableName\n        }\n        $Secrets += $secret\n    }\n}\n\nWrite-Verbose \"Secrets to retrieve: $($Secrets.Count)\"\nWrite-Verbose \"Print variables: $PrintVariableNames\"\n\n$retrievedSecrets = @{}\n\n# Retrieve Secrets\nforeach ($secret in $secrets) {\n    $name = $secret.Name\n    $versionId = $secret.SecretVersionId\n    $versionStage = $secret.SecretVersionStage\n    $variableNameOrPrefix = $secret.variableNameOrPrefix\n    $keys = $secret.Keys\n    \n    # Should we extract only specified keys, or all values?\n    $SpecifiedKeys = $True\n    if ($keys.Count -eq 1 -and ($keys[0] -ieq \"all\" -or $keys[0] -ieq \"*\")) {\n        $SpecifiedKeys = $False\n    }\n    \n    $displayName = Format-SecretName -Name $name -VersionId $versionId -VersionStage $versionStage -Keys $keys\n    Write-Verbose \"Retrieving Secret $displayName\"\n    $_secretIdentifier = \"$name\"\n\n    $params = @(\"--secret-id $name\")\n    if (![string]::IsNullOrWhiteSpace($versionId)) {\n        $params += \"--version-id $versionId\"\n        $_secretIdentifier += \"_$versionId\"\n    }\n    if (![string]::IsNullOrWhiteSpace($versionStage)) {\n        $params += \"--version-stage $versionStage\"\n        $_secretIdentifier += \"_$versionStage\"\n    }\n    \n    # Check to see if we've already retrieved this secret value to save on requests\n    if (-not $retrievedSecrets.ContainsKey($_secretIdentifier)) {\n        $command = \"aws secretsmanager get-secret-value $($params -Join \" \")\"\n        Write-Verbose \"Invoking command: $command\"\n        $response = Invoke-Expression -Command $command\n        if ([string]::IsNullOrWhiteSpace($response)) {\n            throw \"Error: Secret $displayName not found or has no versions.\"\n        }\n        Write-Verbose \"Added secret to retrieved collection ($_secretIdentifier)\"\n        $retrievedSecrets.Add($_secretIdentifier, $response)\n    }\n    else {\n        Write-Verbose \"Rehydrating previously stored secret ($_secretIdentifier) instead of calling AWS.\"\n        $response = $retrievedSecrets.$_secretIdentifier\n    }    \n    \n    try {\n        $AwsSecret = $response | ConvertFrom-Json\n        $AwsSecretValue = $AwsSecret.SecretString | ConvertFrom-Json\n        $secretKeyValues = $AwsSecretValue | Get-Member | Where-Object { $_.MemberType -eq \"NoteProperty\" } | Select-Object -ExpandProperty \"Name\"\n    }\n    catch {\n        Write-Error \"Error converting JSON value returned from AWS for $displayName.`n`nIf secret value is stored as JSON in Plaintext (vs Key/value), check contents validity\"\n    }\n    if ($SpecifiedKeys -eq $True) {\n        foreach ($keyName in $keys) {\n            $variableName = $variableNameOrPrefix\n            if ([string]::IsNullOrWhiteSpace($variableName)) {\n                $variableName = \"$($name.Trim())\"\n            }\n            if ($keys.Count -gt 1) {\n                $variableName += \".$keyName\"\n            }\n            if ($secretKeyValues -inotcontains $keyName) {\n                throw \"Key '$keyName' not found in AWS Secret: $name.\"\n            }\n            $variableValue = $AwsSecretValue.$keyName\n            Save-OctopusVariable -Name $variableName -Value $variableValue            \n        }\n    }\n    else {\n        foreach ($secretKeyValueName in $secretKeyValues) {\n            $variableName = $variableNameOrPrefix\n            if ([string]::IsNullOrWhiteSpace($variableName)) {\n                $variableName = \"$($name.Trim())\"\n            }\n            if ($secretKeyValues.Count -gt 1) {\n                $variableName += \".$secretKeyValueName\"\n            }\n            $variableValue = $AwsSecretValue.$secretKeyValueName\n            Save-OctopusVariable -Name $variableName -Value $variableValue\n        }\n    }\n}\n\nWrite-Host \"Created $($script:storedVariables.Count) output variables\"",
    "Octopus.Action.AwsAccount.Variable": "#{AWS.SecretsManager.RetrieveSecrets.Account}",
    "Octopus.Action.Aws.Region": "#{AWS.SecretsManager.RetrieveSecrets.Region}"
  },
  "Category": "AWS",
  "HistoryUrl": "https://github.com/OctopusDeploy/Library/commits/master/step-templates//opt/buildagent/work/75443764cd38076d/step-templates/aws-secrets-manager-retrieve-secrets.json",
  "Website": "/step-templates/5d5bd3ae-09a0-41ac-9a45-42a96ee6206a",
  "Logo": "iVBORw0KGgoAAAANSUhEUgAAAMgAAADICAMAAACahl6sAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAADNQTFRF////9o0R/eLD/Nu0/erS95Qg+bhr95sv/vHh+r96/vjw+bFc/NSl+KI++82W+saI+KpNeDqM1wAAA41JREFUeNrsnG2XazAURiuo0Cr//9feliIvR3DvXJFZe3+a6XpW5+xWEpyY2w0AAAAAAAAAAAAAAAAAAADgf1J0bda/9N70q83a3enzUHWVjbR1sW0xp6sd6fPI72VmUt3zA+kymD6N5vnIBMrHsxHTjsUXOX0e+iVaTNU5Q0A/Q+k+4oAp+ixMbw6A4rGVVjGHR92ulNXWuTAlBNJN/FFyr5yy3qN9rawmF9IxR4hqX4U1WMplmGtruVBDuiuswbKkzaGhX+cfXsqbZlXXv0dsYR13nw9fLenGXD7f6U5Ony4yTpzyZLNMUcpMr0xNzfwdRRMR1/LP2cqMctNqKx1LZFydm2U022ueEtLL6HbHfmSRYRn4HDXaXyzU4XRkkZWK/+JlRBBBBBFEEEEEEUQQQQQRRBBB5B9uYJc7SyuLw+nI7R2ptKWJcywd18Utza0rnM4iN66M6qzS5E93Lf1zLaviUL/ISs/Nt6W00DEyuRgiP2Yxvrd15z/Y26ncG76jy1Ta5jEy/L0p/VMWy33woVm8UYN1Y9fqKrzfZ5iedtaV34+kNxHak2Wg2SSkY7djx/bQWkNP6nkE0lH3Lyx7D1aak1Z1erWJ+U130Vz0Sude7mZqv995nW7mZxJd27Sg5XQppuMdWY3xl1XXOge8MasWjZfund0KbvrkE9fK7OPNne+2U9YEWX3nemtSbvLv6LJ7gZ9X45yBl9ZxrZ9d3vjT8rz62tOsny7jXkpYPX9jQmvF8yF55TdaslGviZy1vAmfoTobsZztGNEv7qZZSr/6HRc/0yzlb3HiKhURRBBBBBFEEEEEEUQQQQQRRBD5XSLav38tllbVzeH02Ww/UWA+6XgsHdXFKc2vK5Quoz/duVRnlrb26crpizzXOVU3l2Zb5Pfe+d1OX8ViqW7qH9gt51K44bukr2XxrW54vMaoy7mxa/cgvPRVKcQG7uOCD58HLQLt3r17Iy6AqjYeDG7TUenWW+p9Ot/IOF/lwuHV1nk6o8M469PWXhtr+0BeX/x7Ue40W3xacfb2gXFxUZcX8TYB3Kyfp+GThsjKti2zgZuMiLshxW3gpiQyrn/DXhR/i1NqIte5pkUEEUQQQQQRRBBBBBFEEEEEEUR+g4jQUZBEqjqFO9mOiyeShoXvYoukZOG4GCLpWZgu83/vTNRidhlE0rYAAAAAAAAAAAAAAAAAAACAZPkjwAAMDi+bsnPP/wAAAABJRU5ErkJggg==",
  "$Meta": {
    "Type": "ActionTemplate"
  }
}

History