Can't access GCP Access Secret Manager from Dataproc sparkjob

Question

I am trying to fetch GCP secret manager secret from dataproc spark job. But I am getting the error "Exception in thread "main" java.lang.NoClassDefFoundError: com/google/cloud/secretmanager/v1/AccessSecretVersionResponse". I have added the jars "google-cloud-secretmanager-1.4.2.jar" and "gax-1.62.0.jar" in the dataproc spark job dependency.

I am using the code mentioned in the below GCP link. https://cloud.google.com/secret-manager/docs/reference/libraries

Am I missing something here?

I am using the code mentioned in the below GCP link. https://cloud.google.com/secret-manager/docs/reference/libraries

Am I missing something here?

user_13723236 · Answer

2.0-debian10 has python >= 3.0 installed. google-cloud-secretmanager-1.4.2.jar does not support python >= 3.0 (https://pypi.org/project/google-cloud-secret-manager/1.0.0/). Please use a later version of google-cloud-secretmanager.

Can't access GCP Access Secret Manager from Dataproc sparkjob

1 Answers