I want to be able to set the following env variables while submitting a job via dataproc submit:
- SPARK_HOME
- PYSPARK_PYTHON
- SPARK_CONF_DIR
- HADOOP_CONF_DIR
How can I achieve that?
How to pass env variables in dataproc submit command?
Viewed 417
1 Answers
Check the doc Setting environment variables on Dataproc cluster nodes on how to set env variables for different components in Dataproc.
Related