AWS Step Function: Pass array type value to Task Parameter

user_12659025 0
Viewed 1562

Background:

I'm creating a Terraform module that provisions an AWS Step Function machine. The Step Function definition consists of a Map state that iterates over the dynamic array input via a CodeBuild Task state. Within the EnvironmentVariablesOverride parameter of the CodeBuild task, I'm passing the respective Map iteration of the array input.

Problem:

I haven't figure out the syntax for providing the EnvironmentVariablesOverride attribute within the CodeBuild Task Parameter field.

Here's the state machine definition:

resource "aws_sfn_state_machine" "this" {
  name     = <name>
  role_arn = <role_arn>

  definition = <<EOF
{
  "StartAt": "Parallelize Stack",
  "States": {
    "Parallelize Stack": {
      "Type": "Map",
      "End": true,
      "Iterator": {
        "StartAt": "Deploy",
        "States": {
          "Deploy": {
            "Type": "Map",
            "Parameters": {
              "Path.$": "$$.Map.Item.Value"
            },
            "End": true,
            "Iterator": {
              "StartAt": "Plan",
              "States": {
                "Plan": {
                  "Type": "Task",
                  "Resource": "arn:aws:states:::codebuild:startBuild",
                  "Parameters": {
                    "ProjectName": "${var.build_name}",
                    "EnvironmentVariablesOverride.$": "[
                      {
                        "name": "PATH",
                        "type": "PLAINTEXT",
                        "value": "$.Path"
                      },
                      {
                        "name": "COMMAND",
                        "type": "PLAINTEXT",
                        "value": "plan"
                      }
                    ]"
                  },
                  "Next": "Approval"
                },
                "Approval": {
                  "Type": "Task",
                  "Resource": "${aws_sfn_activity.manual_approval.id}",
                  "End": true
                }
              }
            }
          }
        }
      }
    }
  }
}
  EOF
}

Attempts:

  1. Using the EnvironmentVariablesOverride value specified above:
"EnvironmentVariablesOverride.$": "[
    {
        "name": "PATH",
        "type": "PLAINTEXT",
        "value": "$.Path"
    },
    {
        "name": "COMMAND",
        "type": "PLAINTEXT",
        "value": "plan"
    }
]"

InvalidDefinition: Invalid State Machine Definition: 'INVALID_JSON_DESCRIPTION: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in string value

"EnvironmentVariablesOverride.$": States.Format('[
    {
        "name": "PATH",
        "type": "PLAINTEXT",
        "value.$": {}
    },
    {
        "name": "COMMAND",
        "type": "PLAINTEXT",
        "value": "plan"
    }
]', "$.Path")

InvalidDefinition: Invalid State Machine Definition: 'INVALID_JSON_DESCRIPTION: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in string value

"EnvironmentVariablesOverride.$": [
    {
        "name": "PATH",
        "type": "PLAINTEXT",
        "value": "$.Path"
    },
    {
        "name": "COMMAND",
        "type": "PLAINTEXT",
        "value": "plan"
    }
]

InvalidDefinition: Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: The value for the field 'EnvironmentVariablesOverride.$' must be a STRING that contains a JSONPath but was an ARRAY at /States/Parallelize Stack/Iterator/States/Deploy/Iterator/States/Plan/Parameters'

"EnvironmentVariablesOverride.$": "States.JsonToString([
    {
        "name": "PATH",
        "type": "PLAINTEXT",
        "value": "$.Path"
    },
    {
        "name": "COMMAND",
        "type": "PLAINTEXT",
        "value": "plan"
    }
])"

InvalidDefinition: Invalid State Machine Definition: 'INVALID_JSON_DESCRIPTION: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in string value

"EnvironmentVariablesOverride.$": [
    {
    "name": "PATH",
    "type": "PLAINTEXT",
    "value.$": "$.Path"
    },
    {
    "name": "COMMAND",
    "type": "PLAINTEXT",
    "value": "plan"
    }
]

InvalidDefinition: Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: The value for the field 'EnvironmentVariablesOverride.$' must be a STRING that contains a JSONPath but was an ARRAY at /States/Parallelize Stack/Iterator/States/Deploy/Iterator/States/Plan/Parameters'
aws-codebuild aws-step-functions amazon-web-services
2 Answers

This is kind of late, but for people who is going to search for how to start AWS CodeBuild with Environment variables from AWS Step Function. The author's problem is lowercase key names. Should be:

"EnvironmentVariablesOverride": [
  {
    "Name": "PATH",
    "Type": "PLAINTEXT",
    "Value.$": "$.Path"
  },

AWS API documentation https://docs.aws.amazon.com/codebuild/latest/APIReference/API_EnvironmentVariable.html confuses, because gives key names in lowercase

user_112996440

Your attempt number 5 should work if you remove the ".$" so that it goes like below.

"EnvironmentVariablesOverride": [
    {
    "name": "PATH",
    "type": "PLAINTEXT",
    "value.$": "$.Path"
    },
    {
    "name": "COMMAND",
    "type": "PLAINTEXT",
    "value": "plan"
    }
]

I have not tried using terraform (I used Cfn) for this but the state machine definition should be the same regardless of which IaC tool you use. Below is a snippet from my Cfn template (yaml, so you'll have to know convert to json).

  Build:
    Type: Task
    Resource: arn:aws:states:::codebuild:startBuild.sync
    Parameters:
      ProjectName: !Ref ProjectName
      EnvironmentVariablesOverride:
      - Name: stackName
        Type: PLAINTEXT
        Value.$: $.repository_name
      SourceLocationOverride.$: $.artifact_location
    TimeoutSeconds: 300
    ResultPath: null
    Catch:
    - ErrorEquals:
      - States.ALL
      ResultPath: $.error-info
      Next: SendAlert
    Next: SetAccountIdRBITSandpit

EnvironmentVariablesOverride expects a List. But if you add the ".$" then it would expect a String value with a JSONpath.

user_162099980
Related