pyspark

4047 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Is there a way to limit node-wise parallelization of certain memory-intensive UDF computations in PySpark?
user_106210720
•
2
0
101
pyspark apache-spark
Pyspark - java.lang.OutOfMemoryError: Java heap space while writing to csv file
user_122177530
•
2
2
1538
pyspark apache-spark python-3.x
PySpark UDF optimization challenge using a dictionary with regex's (Scala?)
user_4396210
•
4
3
1331
pyspark user-defined-functions python-3.x scala regex
I'm getting an unexpected failed assertion error when joining Spark Dataframe - Found duplicate rewrite attributes
user_75777300
•
5
1
3050
pyspark databricks apache-spark
How to remove double quotes from column name while saving dataframe in csv in spark?
user_133876560
•
2
1
2802
pyspark apache-spark dataframe csv
How to read a checkpointed DataFrame in PySpark
user_65151100
•
2
0
133
checkpoint pyspark
how to detect if your code is running under pyspark
user_1497410
•
4
1
590
pyspark
Spark java DataFrame Date filter based on max Date another DataFrame
user_140824090
•
2
2
453
pyspark apache-spark-sql apache-spark java
Pyspark groupBy and consolidatng on multiple distinct column values
user_129680070
•
2
2
201
pyspark apache-spark-sql apache-spark python
how to drop duplicates but keep first in pyspark dataframe?
user_82360290
•
2
1
2701
pyspark apache-spark python
Hot Questions