apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Pyspark Order By 1.5B Records With 20 Distinct Values - Performance Issue
user_152850050
•
1
1
39
pyspark apache-spark-sql apache-spark
Why the PySpark Dataframe's data gets shuffled when I save it as a parquet file?
user_185120670
•
1
0
72
pyspark apache-spark dataframe python
How to properly downscale a spark cluster using dynamic allocation?
user_120486000
•
1
0
153
spark-streaming apache-spark kubernetes
How to resolve "java.lang.ClassNotFoundException" when running Scala Spark project using databricks-connect?
user_98857470
•
1
0
84
databricks-connect databricks apache-spark scala exception
Spark Sql Array contains on Regex - doesn't work
user_104545620
•
1
1
559
apache-spark-sql apache-spark scala sql
How to take only first instance of data in Spark Structured Streaming according to multiple fields?
user_118661430
•
2
0
46
spark-structured-streaming pyspark apache-spark
Understanding Spark broadcasting
user_51918080
•
1
0
62
pyspark apache-spark python
Reading data from sql server in databricks(pyspark)
user_177188810
•
1
1
332
pyspark azure-databricks databricks apache-spark-sql apache-spark
How to change dataframe schema based on JSON string?
user_103344150
•
1
1
108
pyspark apache-spark dataframe python json
How to remove duplicates from DataFrame in Spark basing on particular columns?
user_152473510
•
1
2
72
apache-spark dataframe scala
Hot Questions