apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Window spec/function perform optimum way or any alternative should be preferred
user_32520970
•
2
0
126
spark-cassandra-connector apache-spark-sql apache-spark cassandra scala
Error while making the call to AWS S3 bucket from Pyspark . AWS Error Code: null, AWS Error Message: Bad Request
user_49241190
•
2
1
1878
pyspark apache-spark amazon-web-services amazon-s3
Flatten nested json in Scala Spark Dataframe
user_37743230
•
5
1
8905
apache-spark-sql apache-spark multidimensional-array scala
Getting error while setting pyspark environment
user_131433470
•
3
1
2406
pyspark apache-spark
AWS Glue Job that has bookmarking enabled fails with “Datasource does not support writing empty or nested empty schemas”
user_135644840
•
2
1
812
parquet aws-glue apache-spark
PySpark dataframe Pandas UDF returns empty dataframe
user_80756190
•
4
1
1136
pyspark pandas apache-spark dataframe
What problems can arise from a Spark non-deterministic Pandas UDF
user_18154860
•
8
1
3775
pyspark pandas apache-spark-sql apache-spark python
Why does Spark NOT create partitions based on Parquet block size on read? (Instead it appears to partition by Parquet file compressed size)
user_42120300
•
2
1
1681
parquet apache-spark-sql apache-spark hadoop
Job 65 cancelled because SparkContext was shut down
user_111172550
•
6
1
4362
apache-zeppelin pyspark apache-spark-sql apache-spark hadoop
SPARK mapToPair and saveToTextFile should i use repartition to optimize performance
user_8874210
•
2
2
172
hadoop-yarn apache-spark performance
Hot Questions