apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How should I write unit tests in Spark, for a basic data frame creation example?
user_63505370
•
5
2
9513
apache-spark intellij-idea scala unit-testing
Spark 2.2 Structured Streaming - duplicate check without watermark - controlling the duplicates state size
user_19423460
•
3
0
969
spark-structured-streaming apache-spark duplicates
Spark with BloomFilter of billions of records causes Kryo serialization failed: Buffer overflow.
user_67780480
•
5
1
825
bigdata apache-spark bloom-filter scala
Use groupby or aggregate to merge items in each transaction in RDD or DataFrame to do FP-growth
user_83500610
•
3
2
1783
rdd pyspark apache-spark-sql apache-spark python
where does df.cache() is stored
user_10653580
•
13
4
29811
apache-spark-sql apache-spark
Spark SQL converting string to timestamp
user_50512430
•
21
3
74255
apache-spark-sql apache-spark sql
Spark Task Memory allocation
user_22927910
•
6
1
3891
spark-streaming apache-spark
Compare two Spark dataframes
user_82685560
•
22
7
64498
apache-spark-sql apache-spark
Spark Data frame search column starting with a string
user_80448520
•
15
2
26805
apache-spark-sql apache-spark
Escape commas inside values while writing spark dataframes to csv
user_15216720
•
3
2
3038
spark-dataframe apache-spark csv
Hot Questions