Dataproc: SSL certificate not found for Sqoop job connecting to external PostgreSQL

user_9882754 0
Viewed 214

I need to connect to PostgreSQL db via SSL. I received 2 certificates and 1 key -> sslrootcert=root.crt sslcert=postgresql.crt and sslkey=postgresql.key.der

Here is my import config from Dataproc:

import
-Dmapreduce.job.user.classpath.first=true
-Dhadoop.security.credential.provider.path=jceks://hdfs/secrets/prod/db.jceks
--connect=jdbc:postgresql://xxx.x.x/DBNAME?sslrootcert=root.crt&sslcert=postgresql.crt&sslkey=postgresql.key.der
--username=user1
--table=db1
--target-dir=gs://bucket
--delete-target-dir
--as-avrodatafile
--password-alias=password

The Dataproc coundn't find my SSL certificates: Error: java.lang.RuntimeException: java.lang.RuntimeException: org.postgresql.util.PSQLException: Could not open SSL certificate file postgresql.crt

How I could add certificates and key to dataproc to be sure that these certs be accessible in connection config.

Where I should store SSL certificates to use them in import config during run job :

--connect=jdbc:postgresql://xxx.x.x/DBNAME?sslrootcert=/???/??/root.crt&sslcert=/???/??/postgresql.crt&sslkey=/???/??/postgresql.key.der
sqoop google-cloud-dataproc ssl postgresql
1 Answers

I think you can add the server certificate or its root CA certificate to Dataproc VMs at ${JAVA_HOME}/jre/lib/security/cacerts, for Dataproc it is /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/jre/lib/security/cacerts. You can use an init action or create a custom image to put the cert in the dir.

See this doc and this question for more details.

user_4042640
Related