How to provide the relative path of certificates packaged inside jar?

Viewed 92

I have a scala-spark streaming application consuming kafka TOPICS .

my kafka params looks like this

val kafkaParams = Map[String, Object](
    Bootstrap_servers -> "270618767-1-1328258624:9093",
    Key_deserializer -> classOf[StringDeserializer],
    Value_deserializer -> classOf[StringDeserializer],
    Group_id -> config.Group_id,
    Auto_offset_reset -> "earliest",
    Enable_auto_commit -> (false: java.lang.Boolean),
    Security_protocol -> "SSL",
    SSL_truststore_location-> "src/main/resources/cert/trust.jks",
    SSL_truststore_password -> "****",
    SSL_keystore_location -> "src/main/resources/cert/keystore.jks",
    SSL_keystore_password -> "*****",
    SSL_key_password -> "****",
    SSL_KEYSTORE_TYPE_CONFIG -> "JKS",
    SSL_TRUSTSTORE_TYPE_CONFIG -> "JKS",
    //SSL_KEYSTORE_CERTIFICATE_CHAIN_CONFIG -> scala.io.Source.fromResource("/cert/keystore.jks").mkString,
    //SSL_TRUSTSTORE_CERTIFICATES_CONFIG -> scala.io.Source.fromResource("/cert/net.jks").mkString,
  )

This setup is working fine in intellij with project structure..

enter image description here

but upon submitting the job into dataproc cluster i am getting java.nio.file.NoSuchFileException:

i have sparkVersion = "2.4.8" , scalaVersion := "2.12.10" and running sbt 1.5.5 with addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5")

i am running this assembly into /opt/projectname/lib/assembly.jar folder in dataproc.

0 Answers
Related