apache-spark-sql

2154 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to avoid excessive shuffles in join operation in pyspark?
user_53527480
•
4
2
1967
pyspark apache-spark-sql apache-spark python
FileNotFoundException: Spark save fails. Cannot clear cache from Dataset[T] avro
user_14173800
•
2
2
1840
spark-avro apache-spark-dataset apache-spark-sql apache-spark caching
from_utc_timestamp not taking daylight saving time into account
user_134130660
•
3
1
1684
pyspark apache-spark-sql
UnsatisfiedLinkError while writing to S3 using Staging S3A Committer on Windows
user_78663770
•
2
2
2402
apache-spark-sql apache-spark hadoop amazon-s3 windows
Calculate new column in spark Dataframe, crossing a tokens list column in df1 with a text column in df2 with pyspark
user_135000100
•
2
1
774
pyspark apache-spark-sql apache-spark
Sum the values on column using pyspark
user_56270230
•
3
1
563
pyspark apache-spark-sql
Spark Structured Streaming: NullPointerException when joining two streams
user_71089470
•
2
0
524
apache-spark-2.0 spark-streaming spark-structured-streaming apache-spark-sql apache-spark
Validate date format in a dataframe column in pyspark
user_58868280
•
2
1
8034
pyspark apache-spark-sql apache-spark
Unable to drop a column that I used to explode an array
user_78027010
•
2
1
473
pyspark-dataframes pyspark apache-spark-sql
How to extract file extension from string scala
user_117427720
•
4
1
480
apache-spark-sql scala
Hot Questions