pyspark

4047 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to use Koalas in Dataproc from a local Jupyter Notebook
user_123839610
•
2
1
138
spark-koalas jupyter-notebook google-cloud-dataproc pyspark google-cloud-platform
Koalas dataframe apply lambda two columns of different type
user_81388580
•
2
0
121
spark-koalas bigdata pyspark pandas python
PySpark runs in YARN client mode but fails in cluster mode for "User did not initialize spark context!"
user_80896950
•
3
1
681
dataproc hadoop-yarn google-cloud-dataproc pyspark apache-spark
Running Python app on Spark with Conda dependencies
user_21195220
•
1
0
391
conda-pack conda pyspark apache-spark
Explode multiple columns from nested JSON but it is giving extra records
user_145343490
•
1
1
596
pyspark apache-spark-sql apache-spark nested json
Pyspark: How to check whether a file path with wild character exists in s3
user_119419020
•
2
1
518
pyspark apache-spark amazon-web-services amazon-s3
Changing Spark UI dashboard host and port on SparkContext config (SparkConf)
user_130600470
•
2
1
84
pyspark apache-spark python
How order by with IN clause is works in pyspark?
user_32975470
•
2
1
36
pyspark apache-spark-sql
Pass custom command line arguments to jar via pyspark (rewriting spark-submit call to pyspark)
user_86507840
•
1
0
196
pyspark apache-spark
How can I have nice file names & efficient storage usage in my Foundry Magritte dataset export?
user_35074650
•
3
3
352
foundry-data-connection foundry-python-transform foundry-code-repositories pyspark palantir-foundry
Hot Questions