apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Pyspark Pivot based on column values combinations
user_15399120
•
1
1
377
pyspark apache-spark-sql apache-spark
pyspark join multiple conditon and drop both duplicate column
user_109702020
•
1
1
361
pyspark apache-spark-sql apache-spark python
Regular expressions in Pyspark
user_102542160
•
2
2
180
pyspark apache-spark-sql apache-spark
How can I change the nullable property of a nested struct field in pyspark?
user_43738050
•
1
0
93
pyspark apache-spark python
All tasks finish but dataproc spark job does not conclude
user_80896950
•
2
0
235
dataproc google-cloud-dataproc pyspark apache-spark
Can some explain what is sc._jvm and sc._jsc in spark? For what purpose these 2 are used in spark?
user_175138420
•
1
1
548
bigdata apache-spark jvm java
Performing two count functions on an RDD in pyspark
user_172633290
•
1
2
200
rdd pyspark apache-spark python
Pyspark: aggregate mode (most frequent) value in a rolling window
user_111289230
•
1
2
1093
rolling-computation pyspark apache-spark-sql apache-spark group-by
Spark process data more than available memory
user_75359750
•
1
1
42
pyspark apache-spark
Convert multiple columns in pyspark dataframe into one dictionary
user_107813510
•
1
1
283
pyspark apache-spark-sql apache-spark user-defined-functions python
Hot Questions