I am currently using Cloud Build for my Dataflow Flex template to kick off jobs. Here's my current command:
gcloud beta builds submit --config run.yaml --substitutions _REGION=$REGION \
--substitutions _FMPKEY=$FMPKEY --no-source
Currently this is running fine from Cloud Shell. But now I want the build to be kicked off based on a trigger.. So I created a Cloud Build that will trigger running this file based on dropping a message to a topic:
https://github.com/mmistroni/GCP_Experiments/blob/master/dataflow/pipeline/run.yaml
However, after publishing a message to the selected topic, all my builds fail with the following error:
our build failed to run: generic::invalid_argument:generic::invalid_argument:
'build.service_account' field cannot be set for triggered builds
I cannot see any logs or details, so it's not clear to me what is going on.. I am guessing it has something to do with the last line in my run.yaml?
options:
logging: CLOUD_LOGGING_ONLY
# Use the Compute Engine default service account to launch the job.
serviceAccount: projects/$PROJECT_ID/serviceAccounts/$PROJECT_NUMBER-compute@developer.gserviceaccount.com
However I see no option for selecting the service account in cloud build. Do I need to set some permissions in IAM?