apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Prefix column with zeroes in spark
user_112086650
•
2
2
156
apache-spark scala
Partitioning in Spark 3.1 with java
user_152605210
•
2
0
99
hadoop-yarn google-cloud-dataproc apache-spark-sql apache-spark
Scala Spark: Parse SQL string to Column
user_60160640
•
2
1
81
apache-spark scala sql
PySpark's "DataFrameLike" type vs pandas.DataFrame
user_39453820
•
8
2
595
pyspark apache-spark-sql apache-spark type-hinting python
How to load partitioned parquet dataset with no partition names (in directory names)?
user_93751020
•
1
1
419
pyspark apache-spark-sql apache-spark
I have a StructType schema. I want to store it in a separate file in the same format and read it in my Spark program from that file
user_164061360
•
2
1
171
apache-spark scala schema
Calculate cumulative sum and average based on column values in spark dataframe
user_165252630
•
2
1
177
pyspark apache-spark python
Pyspark version 3.x, repartition not working as expected for large JSON data
user_22651900
•
2
0
66
pyspark apache-spark hadoop
Getting TypeError in WHEN and OTHERWISE condition statements pyspark
user_165252630
•
1
1
144
pyspark apache-spark python
Is there something like Glue "Bookmark" feature in spark which keeps track at job level?
user_169022660
•
4
0
285
incremental-load spark-streaming aws-glue pyspark apache-spark
Hot Questions