pyspark

4047 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Is there any way to get max value from a column in Pyspark other than collect()?
user_128045160
•
4
1
5764
pyspark apache-spark
pyspark aggregate while find the first value of the group
user_64974600
•
3
1
7138
pyspark apache-spark-sql apache-spark python
How to find the size of a dataframe in pyspark
user_105131510
•
3
3
4445
pyspark apache-spark-sql
Container killed by YARN for exceeding memory limits. 6.0 GB of 6 GB physical memory used while running a pyspark job through dataproc cluster
user_136497010
•
3
1
595
google-cloud-dataproc pyspark jobs google-cloud-platform cluster-computing
Error: Cannot access field element on a value with type ARRAY<STRUCT<element STRING>> in BigQuery
user_136497010
•
2
1
400
google-cloud-dataproc pyspark google-bigquery google-cloud-platform sql
Why results structure change when I change column name in Python spark?
user_14922290
•
2
1
47
pyspark apache-spark python
Hyperopt failed to execute mlflow.end_run() with tracking URI: databricks
user_99200260
•
2
1
604
hyperopt mlflow pyspark azure-databricks databricks
How to find the hierarchy levels of a person(employee,manager etc.) using graphframes in pyspark?
user_82739340
•
2
1
642
graphframes pyspark apache-spark breadth-first-search python
spark writeStream into kafka - Difference between awaitTermination() vs awaitAnyTermination()
user_74866210
•
3
1
946
spark-structured-streaming pyspark apache-kafka apache-spark
Date difference in years in PySpark dataframe
user_94503350
•
2
3
6562
pyspark apache-spark-sql apache-spark datediff python
Hot Questions