google-cloud-dataproc

423 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Can run code in pyspark shell but the same code fails when submitted with spark-submit
user_80896950
•
2
1
338
spark-submit hadoop-yarn google-cloud-dataproc pyspark apache-spark
How are spark jobs submitted in cluster mode?
user_80896950
•
2
1
159
spark-submit dataproc google-cloud-dataproc pyspark apache-spark
Do env variables transfer from driver to workers?
user_80896950
•
2
0
266
dataproc google-cloud-dataproc pyspark apache-spark
How to pass env variables in dataproc submit command?
user_80896950
•
2
1
417
spark-submit dataproc google-cloud-dataproc pyspark apache-spark
Is it feasible to enable jupyterlab UI through DataprocClusterCreateOperator
user_101240830
•
1
1
48
airflow google-cloud-dataproc python
Dataproc how to run a initialization-actions script only on master node and skip running on worker nodes
user_123873330
•
2
1
250
hadoop-yarn google-cloud-dataproc apache-spark google-cloud-platform initialization
Run a Data Fusion pipeline only when a file exist
user_177431710
•
3
1
376
google-cloud-data-fusion google-cloud-dataproc etl google-cloud-platform
Facing Issue with DataprocCreateClusterOperator (Airflow 2.0)
user_101240830
•
2
1
710
airflow-2.x google-cloud-composer airflow google-cloud-dataproc python
Why is adding org.apache.spark.avro dependency is mandatory to read/write avro files in Spark2.4 while I'm using com.databricks.spark.avro?
user_47033590
•
3
1
340
spark-avro google-cloud-dataproc apache-spark scala
writing from a Spark DataFrame to BigQuery table gives BigQueryException: Provided Schema does not match
user_27400540
•
4
0
360
google-cloud-dataproc google-bigquery google-cloud-platform
Hot Questions