apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to debug a Spark job on Dataproc?
Admin1
•
6
1
984
google-cloud-dataproc apache-spark google-cloud-platform
Cannot trigger skew join optimization of AQE in spark 3.0.0
user_138481260
•
2
1
198
apache-spark
pyspark parent child recursive on same dataframe
user_133554670
•
2
0
260
pyspark-dataframes pyspark apache-spark-sql apache-spark scala
pandas df to spark df conversion takes long time on Databricks notebook
user_115213870
•
2
1
1381
pyspark pandas azure-databricks databricks apache-spark
How do you call multiple writeStream operations within a single Spark Job?
user_96677210
•
2
2
1020
spark-structured-streaming pyspark apache-spark
How to resolve the eviction of cached table partition metadata from memory while re-partitioning huge data from S3 using Spark?
user_108347880
•
3
0
1269
pyspark apache-spark hive hadoop python
In GCP Dataproc, what is the maximum number of worker nodes we can use in a cluster?
user_114123120
•
3
1
752
google-cloud-dataproc apache-spark distributed-computing google-cloud-platform distributed
Check if External Shuffle Service is enabled on YARN Cluster
user_59278430
•
2
0
464
hadoop-yarn apache-spark
Kafka consumer consuming messages multiple times when trying to process the messages using Spark
user_21110090
•
2
1
418
kafka-consumer-api apache-kafka apache-spark hadoop scala
Spark caching effect on Optimized Logical Plan
user_69339930
•
4
1
284
apache-spark caching
Hot Questions