apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to calculate the optimal size of the small table for broadcast join in spark
user_4849990
•
1
0
62
apache-spark-sql apache-spark
Join on items inside an array column in pyspark dataframe
user_141924590
•
2
1
260
pyspark apache-spark-sql apache-spark dataframe
My S3 bucket has more than 100k parquet files. What is the best way to programmatically merge all the parquet files and make one big parquet
user_87084790
•
1
1
411
parquet pandas apache-spark dataframe amazon-s3
Join two dataframes and replace the original column values using Spark Scala
user_13251090
•
1
1
330
apache-spark-sql apache-spark scala
Pyspark parse datetime field with day and month names into timestamp
user_44621610
•
2
1
715
pyspark apache-spark-sql apache-spark timestamp date
Update nested struct in spark dataset from another struct column
user_98453660
•
1
1
174
apache-spark-sql apache-spark dataframe dataset json
Structured Streaming with Apache Spark coded in Spark.SQL
user_155206150
•
2
1
256
spark-structured-streaming pyspark azure-databricks databricks apache-spark
Match dataframe rows with row-wise pattern conditions and add array column
user_39550220
•
1
1
105
pyspark apache-spark-sql apache-spark python
Missing data when ordering Pyspark Window
user_112218970
•
3
1
72
pyspark apache-spark-sql apache-spark
How to extract multiple columns from one string column using sql
user_153608420
•
2
1
95
apache-spark-sql apache-spark mysql sql
Hot Questions