Are there pros/cons, or maybe different use cases for using spark-submit to submit a python script vs. simply running a .py file with the python executable (and importing SparkSession), like this?
from pyspark.sql import SparkSession
spk = SparkSession.builder.master(master).getOrCreate()
Basically, are there any differences running the script via python and not spark-submit.