apache-spark-sql

2154 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Reading a zst archive in Scala & Spark: native zStandard library not available
user_48247310
•
2
1
1609
zstd apache-spark-sql apache-spark hadoop scala
Is there a way to define pure SQL UDF's that still take advantage of all of Pyspark's optimizations?
user_64324290
•
1
1
92
pyspark apache-spark-sql apache-spark user-defined-functions
Athena type INT64 in parquet is incompatible with type double defined in table schema
user_97774930
•
2
0
1132
amazon-athena aws-glue pyspark apache-spark-sql
Apache Spark and Hudi: tons of output files
user_156360520
•
2
2
760
apache-hudi pyspark apache-spark-sql apache-spark
Count unique values in a row
user_27535010
•
1
1
49
pyspark apache-spark-sql apache-spark row unique
Reading multiple CSV files in Spark and make a DataFrame
user_153819470
•
1
1
4426
pyspark apache-spark-sql apache-spark python
The difference on reading files in PySpark between reading the whole directory then filtering and reading a part of the directory?
user_137034370
•
3
2
349
pyspark apache-spark-sql apache-spark
Pyspark SQL dataframe map with multiple data types
user_66372570
•
4
2
914
aws-glue-spark aws-glue pyspark apache-spark-sql dataframe
Spark deserialization of dataframe struct inside array of struct
user_33116160
•
1
1
142
parquet apache-spark-sql apache-spark
How to sum every N rows over a Window in Pyspark?
user_146151500
•
2
2
1053
pyspark apache-spark-sql apache-spark window-functions
Hot Questions