Trying to create SparkConf using PySpark but getting an error
Code
from pyspark.python.pyspark.shell import spark
from pyspark import SparkConf, SparkContext
from pyspark.shell import sqlContext
from pyspark.sql import SparkSession
conf = SparkConf().setAppName("Test-1 ETL").setMaster("local[*]").set("spark.driver.host", "localhost").set("spark.sql.execution.arrow.pyspark.enabled", "true")
sc = SparkContext(conf=conf)
Error
org.apache.spark.SparkException: Invalid Spark URL: spark://HeartbeatReceiver@xxxx_LPT-324:51380
I have also set set SPARK_LOCAL_HOSTNAME=localhost
Can anyone please help me?