pyspark - can PYTHONPATH be used for the Python interpreter on worker nodes to find Python modules?

Please advise where I should look into to understand the in-detail mechanism on how PySpark finds Python modules on the worker nodes, especially the usage of PHTHONPATH.

PYTHONPATH variable

Environment Variables says environment variables defined in spark-env.sh could be used.

Certain Spark settings can be configured through environment variables, which are read from the conf/spark-env.sh script in the directory where Spark is installed

Then if I define PHTHONPATH in spark-env.sh on all the worker nodes, will PySpark start a Python interpreter process with the PYTHONPATH being passed to the UNIX process, and Python modules will be loaded from the PYTHONPATH?

Precedence of PYTHONPATH when --archives is used

In case PHTHONPATH in spark-env.sh can be used, then what will happen when --archives specifies the virtual environment package?

Python Package Management says a conda environment can be packaged into a tar.gz and passed to the worker nodes.

There are multiple ways to manage Python dependencies in the cluster:

• Using PySpark Native Features
• Using Conda
• Using Virtualenv
• Using PEX

Using Conda

conda create -y -n pyspark_conda_env -c conda-forge pyarrow pandas conda-pack
conda activate pyspark_conda_env
conda pack -f -o pyspark_conda_env.tar.gz

After that, you can ship it together with scripts or in the code by using the --archives option or spark.archives configuration (spark.yarn.dist.archives in YARN). It automatically unpacks the archive on executors.

export PYSPARK_DRIVER_PYTHON=python # Do not set in cluster modes.
export PYSPARK_PYTHON=./environment/bin/python
spark-submit --archives pyspark_conda_env.tar.gz#environment app.py

However when PYTHONPATH is defined in spark-env.sh, will PYTHONPATH be used and which will take precedence either inside the conda environment or PYTHONPATH?