How to pass mixed parameters for an AWS Step Functions state machine

user_10581944 0
Viewed 2325

I have an AWS Step Functions state machine defined in a json file, in step1 (a lambda task), I saved three parameters in the ResultPath:

   "NeededParameters": {
      "year": "2021",
      "month": "04",
      "day": "27"
    },

In step2 (also a lambda task), I understand that if I do something like:

"Step 2" : {
         "Type": "Task",
         "Resource": "arn:aws:states:::lambda:invoke",
         "Parameters": {
             "FunctionName":"${lambda_name}",
             "Payload":{
               "year.$": "$.NeededParameters.year"
             }
         },
         "ResultPath": "$.Step2",

"year.$": "$.NeededParameters.year" will pass 2021 to payload Year.

But what I want to achieved is to define an Amazon S3 path which includes the three parameters, something like:

 "Step 2" : {
          "Type": "Task",
          "Resource": "arn:aws:states:::lambda:invoke",
          "Parameters": {
              "FunctionName":"${lambda_name}",
              "Payload":{
                "s3path.$":"${s3_bucket_name}/$.NeededParameters.year/$.NeededParameters.month/$.NeededParameters.day"
              }
          },
          "ResultPath": "$.Step2",

s3_bucket_name is defined in Terraform resource "aws_sfn_state_machine" "sfn_state_machine" and pass to the statemachine definition json file above, but when I apply terraform it complains:

InvalidDefinition: Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: The value for the field 's3path.$' must be a valid JSONPath at /States/Process Scores/Parameters'

if I change s3path.$ to s3path it still not working, does anyone know how to resolve this? Thanks.

terraform-provider-aws aws-step-functions terraform amazon-web-services json
3 Answers

  1. As the error message implies, the string you pass to s3path.$ is not valid JSONPath. If you want to pass some static value, you need to name it without .$ at the end (simply s3path), otherwise, like in your case, it will be treated and validated as a JSONPath.

  2. Static params don't support any kind of string expansion to my knowledge, especially involving JSONPath. I would suggest passing param called s3BucketName in addition to year, month and day, and then simply construct S3 URL inside lambda function itself.

user_113055810

You could do something like this:

{
   "Step 2" : {
          "Type": "Task",
          "Resource": "arn:aws:states:::lambda:invoke",
          "InputPath": "$.NeededParameters",
          "Parameters": {
              "FunctionName":"${lambda_name}",
              "Payload":{
                "s3path.$": "States.Format('{}/{}/{}/{}', ${s3_bucket_name}, $.year, $.month, $.day)"
              }
          },
          "ResultPath": "$.Step2"
}
user_45260190

To answer @wawawa:

Hi thanks, for the 2nd point, 'construct S3 URL inside lambda function' I guess I can do that but that probably won't work because I want to have a dynamic s3 url (pass the current date everyday). For the first point, if I define as s3path and without $ then I can't pass the values from $.NeededParameters.xxx, I'm a bit stuck on this, or maybe I'm in the completely wrong direction...

You can use the Intrinsic Function of the states language, called States.Format: https://states-language.net/#appendix-b

user_134818760
Related