apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
kafka integration with Pyspark structured streaming (Windows)
user_162988600
•
2
1
2013
spark-kafka-integration pyspark apache-spark python
What is the most efficient way to read in a partitioned parquet file with pyspark?
user_162990230
•
2
0
351
pyspark databricks apache-spark amazon-s3
Spark Cassandra connection through java client
user_162295160
•
2
2
199
scylla datastax apache-spark-sql apache-spark cassandra
Spark operator vs spark submit - how to choose which one to use?
user_95747750
•
3
1
640
apache-spark kubernetes
What's significant change in spark 3.1.2 from 3.0.2 that causes Disk Space Used while enough Memory is available
user_162885610
•
3
1
136
apache-spark
cleanSource option does not delete any files
user_115842710
•
2
1
641
delta-lake spark-structured-streaming apache-spark
Pyspark convert a Column containing strings into list of strings and save it into the same column
user_102195320
•
2
1
549
pyspark apache-spark-sql apache-spark
when we call persist on dataframe and specify storage level to be MEMORY_ONLY or MEMORY_ONLY_SER what is actually serialized/de-serialized?
user_78381690
•
2
0
159
apache-spark dataframe serialization java
How Spark achieves memory fairness between tasks?
user_78381690
•
4
1
334
apache-spark memory-management multithreading java
ClassNotFoundException while decomissioning executor's rdd blocks via SIGPWR (graceful decommision)
user_27930160
•
2
0
103
apache-spark
Hot Questions