rdd

162 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Best approach to transform Dataset[Row] to RDD[Array[String]] in Spark-Scala?
user_131498060
•
3
3
518
apache-spark-dataset rdd apache-spark-sql apache-spark scala
Why "collect" action in spark triggers data collection to driver?
user_52569830
•
1
1
350
rdd pyspark apache-spark-sql apache-spark hadoop
AWS Glue RDD.saveAsTextFile() raises Class org.apache.hadoop.mapred.DirectOutputCommitter not found
user_9136240
•
3
3
798
rdd aws-glue apache-spark scala
getPersistentRDDs returns Map of cached RDDs and DataFrames in Spark 2.2.0, but in Spark 2.4.7 - it returns Map of cached RDDs only
user_76184820
•
2
1
189
rdd apache-spark scala
Differences between persist(DISK_ONLY) vs manually saving to HDFS and reading back
user_87880710
•
3
1
530
rdd apache-spark
How to convert text log which contains partially json string to the structured in pyspark?
user_142804760
•
2
2
179
rdd pyspark apache-spark-sql apache-spark python-3.x
Difference in caching RDD vs. caching DataFrame in Spark
user_122089100
•
4
0
93
rdd apache-spark-sql apache-spark
How to use forEachPartition on pyspark dataframe?
user_82295340
•
2
1
2601
rdd pyspark
how spark handles out of memory error when cached( MEMORY_ONLY persistence) data does not fit in memory?
user_141586000
•
3
1
2073
rdd apache-spark out-of-memory partitioning caching
Spark: Replicate each row but with change in one column value
user_141220300
•
3
3
952
rdd apache-spark-sql apache-spark scala
Hot Questions