apache-spark-sql

2154 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Spark window partition function taking forever to complete
user_4098140
•
6
3
2626
apache-spark-sql apache-spark dataframe scala performance
Difference in caching RDD vs. caching DataFrame in Spark
user_122089100
•
4
0
93
rdd apache-spark-sql apache-spark
Unable to connect/submit a job to a remote Yarn cluser
user_108517830
•
2
0
27
hadoop-yarn hdfs apache-spark-sql apache-spark
How to get Weighted Average for a column in pyspark
user_130444040
•
2
1
549
pyspark apache-spark-sql apache-spark python
ValidationFailureSemanticException: Partition spec contains non-partition columns
user_28054820
•
2
1
1405
hiveql apache-spark-sql apache-spark hive
How to implement a custom Pyspark explode (for array of structs), 4 columns in 1 explode?
user_99919470
•
3
1
729
pyspark apache-spark-sql apache-spark python-3.x
How to pass DataSet(s) to a function that accepts DataFrame(s) as arguments in Apache Spark using Scala?
user_98226290
•
2
1
487
apache-spark-dataset apache-spark-sql apache-spark scala
DataType (UDT) v.s. Encoder in Spark SQL
user_89329100
•
3
0
247
apache-spark-encoders apache-spark-dataset apache-spark-sql apache-spark user-defined-types
Kryo encoder v.s. RowEncoder in Spark Dataset
user_89329100
•
4
2
2449
kryo apache-spark-dataset apache-spark-sql apache-spark serialization
Combine dataframes from different stages into one single dataframe[Spark]
user_4098140
•
1
0
82
apache-spark-sql apache-spark dataframe scala
Hot Questions