Apache Airflow - Python Capability Question

Viewed 260

As per the most recent release of Apache Airflow, Airflow 2.0+, and its upgrading documentation found here, Python 3.9 is not supported. However, my understanding is that Python 3.9 is not supported for running Apache Airflow, but what about .py files, scripts, etc. written in and libs, modules, etc. built upon Python 3.9?

Can I download Python 3.8 to run Apache Airflow, and via the Bash Operators run Python files that operate on Python 3.9?

1 Answers

You can use PythonVirtualenvOperator to run python code in any version you'd like. The operator has python_version parameter that you can set.

Example:

virtualenv_task = PythonVirtualenvOperator(
    task_id="virtualenv_python",
    python_callable=callable_virtualenv,
    requirements=["requests==2.25.0"],
    system_site_packages=False,
    python_version="2.7" # set to the version you'd like.
    dag=dag,
)

You can read about it more in the docs

Related