apache-spark

apache-spark

436 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Getting ValidationMetricIdentifier tuple must have at least six components when setting up code based great expectations data context
Pablo Beltran0
•
0
0
14
great-expectations pyspark apache-spark azure
How to remove scientific notation while generating CSV files as output?
userdWEl10
•
0
1
58
databricks pyspark apache-spark azure
Distinct operation in spark structured streaming with a window operation
Deval0
•
0
0
31
spark-streaming-kafka spark-streaming spark-structured-streaming apache-spark
Spark Structured Streaming - unable to change max.poll.records (showing as 1)
Karan Alang0
•
1
0
39
spark-structured-streaming kafka-consumer-api apache-kafka pyspark apache-spark
Best practice when using multiple when in Spark/PySpark
Tarique0
•
0
1
35
conditional-statements dataframe pyspark apache-spark apache-spark-sql
Join two big tables and to get most recent value
Fisseha Berhane0
•
0
2
46
date join pyspark apache-spark apache-spark-sql
Retrieve the valid Party ids from the Latest record DataFrame based on lookup from Invalid Party DataFrame
Silpa Pinjala0
•
1
1
26
apache-spark scala
Spark Job Errors OOM Error Unless I Realize the Dataset
MarkF0
•
0
0
25
apache-spark
Spark Structured Streaming Foreachbatch [ pyspark]
user8454660
•
0
0
18
delta-lake spark-structured-streaming apache-spark
Filter an array Struct column based on a provided list
always_stuck0
•
0
0
51
user-defined-functions apache-spark apache-spark-sql scala
Hot Questions