apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
PySpark Incorrectly Inferring Parquet Date
user_19836820
•
1
0
40
parquet aws-glue pyspark apache-spark-sql apache-spark
Spark Streaming Job starts reading from beginning instead of where it stopped consuming
user_194019620
•
2
0
90
spark-structured-streaming apache-kafka apache-spark mongodb
Attribute error when using SparkContext to parallelize a class function in DataBricks
user_194000150
•
1
0
58
pyspark databricks apache-spark parallel-processing python
How to add empty map<string,string> type column to DataFrame in PySpark?
user_134654360
•
2
2
130
pyspark apache-spark-sql apache-spark python
Avro bytes from Event hub cannot be deserialized with pyspark
user_93744240
•
1
2
138
spark-avro azure-synapse azure-eventhub apache-spark
Running unit tests with Spark 3.3.0 on Java 17 fails with IllegalAccessError: class StorageUtils cannot access class sun.nio.ch.DirectBuffer
user_12129600
•
2
2
1138
java-17 apache-spark java
Pyspark groupby for all column with unpivot
user_146700360
•
1
3
81
pyspark apache-spark-sql apache-spark
Union of Spark DataFrames
user_193065650
•
1
1
60
apache-spark scala
Error while deploy: Class org.apache.hadoop.fs.LocalFileSystem not found
user_184168020
•
1
1
196
apache-spark sbt hadoop
deequ rule to check for gap in numeric sequence
user_2039680
•
2
0
25
amazon-deequ apache-spark
Hot Questions