apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Get length of Impala table varchar columns from Spark
user_132501770
•
1
0
175
impala apache-spark
Avro vs ORC memory consumption/performance in Spark
user_141399680
•
1
0
53
spark-avro orc avro apache-spark
filter files by pyspark date
user_127495190
•
1
3
636
pyspark apache-spark python
Overwrite mode in spark causing issues
user_77609100
•
2
0
454
aws-glue-spark aws-glue pyspark apache-spark amazon-web-services
What is the best way to fill missing info on all columns with Null\0 for missing records in Spark dataframe while groupby?
user_104527000
•
2
2
512
pyspark apache-spark-sql apache-spark
Spark's mapPartition is causing skew among worker's data distribution. Is there a way to map data partition on workers based on partition size?
user_116412770
•
1
0
44
apache-spark java
How to groupBy in Spark using two columns and in both directions
user_133198360
•
1
2
481
apache-spark-sql apache-spark dataframe scala
AWS EMR step Vs command line spark-submit
user_134916440
•
1
1
810
amazon-emr apache-spark
How to remove all the subset from a column except few based on the other column in Pyspark?
user_173637350
•
2
1
47
pyspark apache-spark-sql apache-spark
Collect rows as an array of a Spark dataframe after a group by using PySpark
user_74758790
•
1
1
816
pyspark apache-spark-sql apache-spark
Hot Questions