apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Stream-Static Join: How to refresh (unpersist/persist) static Dataframe periodically
user_120945660
•
5
1
2097
spark-streaming spark-structured-streaming apache-spark-sql apache-spark scala
Dataproc notebook cannot import or export to BigQuery : Class Not Found Exception
user_87152750
•
2
2
245
google-cloud-dataproc apache-spark google-bigquery
How to quickly check if row exists in PySpark Dataframe?
user_137770190
•
3
2
3752
pyspark apache-spark-sql apache-spark exists python
NoSuchMethodError when upgrading spark to 2.4.5
user_129320310
•
2
1
159
apache-spark version upgrade
Create a new column by replacing comma-separated column's values with a lookup based on another dataframe
user_151859020
•
3
2
249
pyspark apache-spark-sql apache-spark python
Apache Airflow to Orchestrate DAG *within* Spark job?
user_63934680
•
2
0
96
airflow pyspark apache-spark
How to trigger google dataproc job using airflow and pass parameter as well
user_100644850
•
2
1
1397
airflow google-cloud-dataproc pyspark apache-spark google-cloud-platform
How to update a dataframe in PySpark with random values from another dataframe?
user_151826540
•
2
2
445
pyspark apache-spark-sql apache-spark dataframe
How to apply groupBy and aggregate functions to a specific window in a PySpark DataFrame?
user_143672790
•
2
1
240
pyspark apache-spark-sql apache-spark python
Compare the dataframe value with the list of values and get the list of values which is not present in dataframe
user_96239000
•
2
3
45
pyspark apache-spark-sql apache-spark python
Hot Questions