PySpark python error: Exception: Java gateway process exited before sending the driver its port number

Viewed 2832

I am using PySpark with python 3 and i just downloaded the pyspark with command

conda install pyspark

and its successfully download, I imported the file in Jupyter Notebook and then I tried to run the following code

sc1 = sp.SparkContext.getOrCreate()

I am getting the following exception

Exception                                 Traceback (most recent call last)
<ipython-input-9-a4daca8dcb9d> in <module>()
----> 1 sc1 = sp.SparkContext.getOrCreate()

~\Anaconda3\lib\site-packages\pyspark\context.py in getOrCreate(cls, conf)
    332         with SparkContext._lock:
    333             if SparkContext._active_spark_context is None:
--> 334                 SparkContext(conf=conf or SparkConf())
    335             return SparkContext._active_spark_context
    336 

~\Anaconda3\lib\site-packages\pyspark\context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
    113         """
    114         self._callsite = first_spark_call() or CallSite(None, None, None)
--> 115         SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
    116         try:
    117             self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,

~\Anaconda3\lib\site-packages\pyspark\context.py in _ensure_initialized(cls, instance, gateway, conf)
    281         with SparkContext._lock:
    282             if not SparkContext._gateway:
--> 283                 SparkContext._gateway = gateway or launch_gateway(conf)
    284                 SparkContext._jvm = SparkContext._gateway.jvm
    285 

~\Anaconda3\lib\site-packages\pyspark\java_gateway.py in launch_gateway(conf)
     93                 callback_socket.close()
     94         if gateway_port is None:
---> 95             raise Exception("Java gateway process exited before sending the driver its port number")
     96 
     97         # In Windows, ensure the Java child processes do not linger after Python has exited.

Exception: Java gateway process exited before sending the driver its port number

I tried few solution, but none of them worked.. Any suggestion??

2 Answers

The solution is provided here Solution for windows user, how to install it, step by step instruction, I followed and it worked for me, try this.

Related