I have an existing, functional Dataproc workflow template, and I am attempting to create a Cloud Scheduler job to run it on a schedule. The workflow template runs without error when run via the "RUN" button on the Dataproc Workflow Template console page.
I created a Scheduler job to run this workflow template with the following command (names redacted here):
gcloud scheduler jobs create http <job-name>
--location=us-central1
--schedule="0 1 * * *"
--http-method=POST
--uri=https://dataproc.googleapis.com/v1/projects/<project-name>/regions/us-central1/workflowTemplates/<template-name>:instantiate?alt=json
--oauth-service-account-email=<service-account-name>@<project-name>.iam.gserviceaccount.com
--oauth-token-scope=https://www.googleapis.com/auth/cloud-platform
The job is successfully created, but upon attempting to run it, it returns INVALID_ARGUMENT. Here is the response:
{"@type":"type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished", "jobName":"projects/<project-name>/locations/us-central1/jobs/<job-name>",
"status":"INVALID_ARGUMENT", "targetType":"HTTP", "url":"https://dataproc.googleapis.com/v1/projects/<project-name>/regions/us-central1/workflowTemplates/<workflow-name>:instantiate?alt=json"}
I get no more descriptive response than that. There is no sign the workflow began, nor any failures listed on the Dataproc Workflow console page.
I also tried a similar Scheduler job with the same service account, scope, location, and schedule, but I cut the URL down to the base workflow location: https://dataproc.googleapis.com/v1/projects/<project-name>/regions/us-central1/workflowTemplates/<template-name> and tried --http-method=GET, and that was successful, but, of course, does not instantiate the workflow.
Is there either something I'm missing, or is there at least a better way to diagnose the issue?