apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Optimize Join of two large pyspark dataframes
user_81620180
•
2
1
2132
pyspark apache-spark-sql apache-spark join
spark creates partition inside partition on S3
user_27535760
•
2
1
65
apache-spark-dataset apache-spark-sql apache-spark amazon-s3 scala
Why Spark Shuffle Spill (memory) is much larger than Shuffle Read Size ? What is the Shuffle Spill (disk)?
user_44550230
•
3
0
191
apache-spark shuffle
Scala Thread Pool - Invoking API's Concurrently
user_5359350
•
4
2
1041
databricks apache-spark blockingqueue scala functional-programming
Run spark sql query in AWS EMR
user_81620180
•
2
0
2534
amazon-emr pyspark apache-spark hive amazon-web-services
PySpark - Add Row Number After Using Explode Function
user_32420360
•
6
1
996
pyspark apache-spark explode row-number
Why does Spark Structured Streaming not allow changing the number of input sources?
user_96677210
•
2
2
1030
spark-structured-streaming pyspark apache-kafka apache-spark
Unable to write data from Spark to Azure Blob Storage - generic Microsoft error message
user_137092280
•
2
0
992
apache-spark azure-storage
Missing Avro Custom Header when using Spark SQL Streaming
user_65300710
•
2
1
664
spark-avro spark-streaming spark-structured-streaming apache-spark-sql apache-spark
Collaborative filtering spark python
user_135352130
•
2
1
225
pyspark apache-spark-sql apache-spark collaborative-filtering json
Hot Questions