How to import external class inside the PythonVirtualEnvOperator Airflow?

Viewed 41

I have a dag like below,

from datetime import datetime
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from airflow.operators.python_operator import PythonVirtualenvOperator
import platform

def get_info():
   from modelsFile import model_config
    print("version: ")
dag = DAG('sample', description='Sample DAG',
          schedule_interval=None,
          start_date=datetime(2022, 5, 1), catchup=False)
get_info_operator = PythonVirtualenvOperator(task_id='get_info_task', python_callable=get_info, dag=dag)

get_info_operator

Since I have used PythonVirtualEnvOperator, I need to give all the dependencies inside the python_callable function. But, when I tried to import the class model_config from the file modelsfile, It is throwing the error as "ModuleNotFoundError: No module named 'modelsfile'".

But, when I change from pythonVirtualEnvOperator to PythonOperator, It is working fine.

Can anyone help me to solve this issue in Airflow PythonVirtualEnvOperator?

0 Answers
Related