How to specify a shielded VM and secure boot for a dataflow job?

user_3143112 0
Viewed 969

The dataflow job run command has several parameters for workers such as worker-machine-type. But I don't see a specification for Shielded VM and Secure Boot. Is there a way to designate these parameters to the dataflow job? The issue is that the "Shielded VM" policy has been set at the organization level so the job fails with

Startup of the worker pool in zone us-west1-a failed to bring up any of the desired 2 workers. The project quota may have been exceeded or access control policies may be preventing the operation; review the Stackdriver Logging \"GCE VM Instance\" log for diagnostics
google-cloud-dataflow google-cloud-platform
3 Answers

At this time, it is not possible to specify Dataflow job parameters to meet your organization constraint.

There is a Issue Tracker for this item:

Support Shielded VMs for the Underlying GCE Instances

I recommend adding yourself to this issue to increase visibility.

[UPDATE 2022-07-07]

Credit to @martin-beck's post providing updated status.

Dataflow Shielded VM

Starting on June 1, 2022, the Dataflow service uses Shielded VM for all workers. To learn more about Shielded VM capabilities, see Shielded VM.

user_80167200
user_6331500
user_173212960
Related