I am using Spark 2.1.0. In this version getting a SparkSession is through using a builder, e.g. SparkSession.builder.app_name().config("some-property", "something-value").
Earlier versions used spark.conf.set("property", "value").
Are this SparkSession.config("property", "value") and spark.conf.set("property", "value") the same? If not, what is the difference?
I also tried:
SparkSession.builder.app_name()
.config("some-property1", "something-value1")
.config("some-property2", "something-value2")
Is this same as the below?
spark.conf.set("property1", "value1")
spark.conf.set("property2", "value2")