AWS glue workflow doesn't wait for parallel jobs

Viewed 10

I have parallel processing (I run two jobs simultaneously), and after these two jobs, I have condition which should wait until two previous parallel jobs will complete. In the AWS glue workflow, I see Trigger logic Start after ANY watched event, and in my opinion, instead of ANY, it should be ALL, but the question is how to do it in a JSON file.

 {
  "Name": "test-job",
  "WorkflowName": "test-workflow",
  "Type": "CONDITIONAL",
  "Predicate": {
    "Conditions": [
      {
        "State": "SUCCEEDED",
        "LogicalOperator": "EQUALS",
        "JobName": "import-table1"
      },
      {
        "State": "SUCCEEDED",
        "LogicalOperator": "EQUALS",
        "JobName": "import-table2"
      }
    ],
    "Logical": "AND"
  },
  "Actions": [
    {
      "JobName": "test-job"
    }
  ],
  "StartOnCreation": true
}
0 Answers
Related