apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to solve an assignment problem (like Hungarian/linear_sum_assignment) with an edge case in PySpark UDF
user_76506350
•
10
1
463
hungarian-algorithm scipy-optimize pyspark apache-spark python
When exactly should I repartition my dataframes?
user_50131330
•
1
0
34
apache-spark dataframe
EMR ignores spark submit parameters (memory/cores/etc)
user_5553550
•
5
2
635
amazon-emr apache-spark amazon-web-services
Pyspark Window function: Counting number of categorical variables and calculating percentages
user_83829500
•
2
2
288
pyspark apache-spark-sql apache-spark
Spark withWatermark, only store values before and after a gap
user_32227820
•
2
0
125
spark-streaming pyspark apache-spark-sql apache-spark
Configuring timezones on Databricks Spark cluster, conflict between Spark configuration and system settings
user_81597770
•
4
0
811
databricks apache-spark timezone time
Are there any drawbacks of using left join of smaller table with larger table vs inner join of two tables and right joining smaller table?
user_158395910
•
1
1
98
pyspark apache-spark-sql apache-spark hive
Keep only the newest data with Spark structured streaming
user_149341810
•
1
1
103
spark-streaming pyspark databricks apache-spark
Spark application syncing with Hive metastore - "There is no primary group for UGI spark" error
user_15812660
•
1
1
337
hive-metastore apache-spark-sql apache-spark kubernetes hive
How to get second highest value from a column pyspark?
user_100376890
•
4
4
1752
pyspark apache-spark dataframe group-by python
Hot Questions