apache-spark-sql

2154 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Databricks - How to join a table with IDs contained in a column of type struct<array<string>>
user_63406090
•
1
1
358
delta-lake pyspark databricks apache-spark-sql apache-spark
Possible to use Spark Pandas UDF in pure Spark SQL?
user_2234780
•
6
3
391
pyspark apache-spark-sql
pandas_udf to extract a value from a column containing maps
user_74758790
•
1
1
137
pyspark pandas apache-spark-sql dataframe user-defined-functions
Create a new dataframe (with different schema) from selected information from another dataframe
user_144766900
•
1
2
262
apache-spark-sql apache-spark scalability scala java
Optimizing "withColumn when otherwise" performance in pyspark
user_76292570
•
2
2
876
pyspark databricks apache-spark-sql apache-spark
Spark history server - find part of SQL query for stage
user_8363180
•
2
0
92
apache-spark-sql apache-spark
Window & Aggregate functions in Pyspark SQL/SQL
user_5381570
•
1
2
77
pyspark apache-spark-sql sql
How to load data at Spark Dataframe in each Worker to prevent loading huge data to Master node
user_66405040
•
2
1
591
apache-spark-sql apache-spark scala
Databricks Delta Lake - Reading data from JSON file
user_63406090
•
2
1
1023
delta-lake pyspark databricks apache-spark-sql apache-spark
AWS EMR: Pyspark: Rdd: mappartitions: Could not find valid SPARK_HOME while searching: Spark closures
user_73439220
•
7
2
520
amazon-emr python-requests pyspark apache-spark-sql apache-spark
Hot Questions