How to configure Spark Standalone's web UI for HTTPS?

user_8072388 0
Viewed 3118

I'd like to set up Spark Standalone's Web UI so it can be accessed through HTTPS.

Spark is being run on a cluster external to the computer I'm using to access the browser.

Here's what I've done so far, and it's not working:

  • Used OpenSSL to make a self-signed certificate and key and then keytool to make a keystore
  • Packaged the certificate and key as a .p12 file
  • In Chrome's settings under Manage Certificates, imported said p12 file

  • In spark-defaults.conf, added options

    spark.ui.https.enabled                   true
spark.ui.ssl.server.keystore.location    /path/to/spark.keystore
spark.ui.ssl.server.keystore.keypassword password
spark.ui.ssl.server.keystore.password    password

  • In spark-env.sh, added

    export SPARK_MASTER_OPTS="-Dspark.ui.https.enabled=true \
  -Dspark.ui.ssl.server.keystore.location=/path/to/spark.keystore \
  -Dspark.ui.ssl.server.keystore.keypassword=password \
  -Dspark.ui.ssl.server.keystore.password=password"

export SPARK_WORKER_OPTS="-Dspark.ui.https.enabled=true \
  -Dspark.ui.ssl.server.keystore.location=/path/to/spark.keystore \
  -Dspark.ui.ssl.server.keystore.keypassword=password \
  -Dspark.ui.ssl.server.keystore.password=password"

I've tried to connect to the server before, after, and in between each of these steps, and I keep getting the error "This site can’t provide a secure connection". What am I missing here?

apache-spark https ssl
2 Answers

Below config worked for me, try putting this in "spark-defaults.conf" and restart the Spark service. Also check logs for which port Spark UI is listening on, as "spark.ssl.ui.port" is set to "0". In my case, it was running on port 8480.

spark.ssl.enabled true
spark.ssl.ui.port 0
spark.ssl.keyStore <path_to_keystore>
spark.ssl.keyStorePassword <keystore_password>
spark.ssl.keyPassword <key_password>
spark.ssl.trustStore <path_to_truststore>
spark.ssl.trustStorePassword <truststore_password>
spark.ssl.enabledAlgorithms ECDHE-RSA-AES256-SHA,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
spark.ssl.protocol TLSv1.2
spark.ssl.trustStoreType JKS

See the screenshot for reference.

Spark Master running on HTTPS

user_53291080
Related