Connecting to remote master on standalone Spark

Viewed 7789

I launch Spark in standalone mode on my remote server via following next steps:

  • cp spark-env.sh.template spark-env.sh
  • append to spark-env.sh SPARK_MASTER_HOST=IP_OF_MY_REMOTE_SERVER
  • and run next commands for standalone mode: sbin/start-master.sh sbin/start-slave.sh spark://IP_OF_MY_REMOTE_SERVER:7077

And I try to connect to remote master:

val spark = SparkSession.builder()
  .appName("SparkSample")
  .master("spark://IP_OF_MY_REMOTE_SERVER:7077")
  .getOrCreate()

And I receive the following errors:

ERROR SparkContext: Error initializing SparkContext.
java.net.BindException: Cannot assign requested address: Service 'sparkDriver' failed after 16 retries!

and warnings:

    WARN Utils: Service 'sparkMaster' could not bind on port 7077. Attempting port 7078.
.....
    WARN Utils: Service 'sparkMaster' could not bind on port 7092. Attempting port 7092.
3 Answers
Related