apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Is there a way to define pure SQL UDF's that still take advantage of all of Pyspark's optimizations?
user_64324290
•
1
1
92
pyspark apache-spark-sql apache-spark user-defined-functions
How to transpose Columns to Rows in a PySpark DataFrame?
user_47250740
•
2
1
2025
pyspark apache-spark transpose pivot
Apache Spark and Hudi: tons of output files
user_156360520
•
2
2
760
apache-hudi pyspark apache-spark-sql apache-spark
Count unique values in a row
user_27535010
•
1
1
49
pyspark apache-spark-sql apache-spark row unique
Adding new column in pyspark dataframe
user_66649900
•
1
1
93
pyspark apache-spark user-defined-functions python
lineSep option with Spark 2.4
user_103166890
•
2
1
1896
google-cloud-storage apache-spark google-cloud-platform scala
Reading multiple CSV files in Spark and make a DataFrame
user_153819470
•
1
1
4426
pyspark apache-spark-sql apache-spark python
The difference on reading files in PySpark between reading the whole directory then filtering and reading a part of the directory?
user_137034370
•
3
2
349
pyspark apache-spark-sql apache-spark
Send email from Databricks Notebook with attachment
user_151125630
•
2
1
8121
pyspark azure-databricks databricks apache-spark email
Pyspark dataframe read from one bucket and write to another bucket with different KMS keys in same job
user_961800
•
2
1
540
amazon-kms pyspark apache-spark amazon-web-services amazon-s3
Hot Questions