Currently am learning airflow to perform my batch processing task in python.
I just created a python file under dag folder in AIRFLOW_HOME path that exist in root directory of my Django rest api project. I tested by creating some sample dags in file that exist in dags folder.
But i want to create airflow dag when one of my Django rest api is called. Basically am getting job metadata from UI and storing it in my database. So i want to create a dag using that data in my Django views file.
I tried by pasting same code in my Django views file that i used in files under dags folder but its not creating dags. Here is what i tried in my Django views file
dag_id = 'test_dag_1'
default_args = {'owner': 'airflow',
'start_date': datetime.datetime(2021, 9, 13)
}
schedule = '@daily'
dag_number = 1
dag = DAG(dag_id,
schedule_interval=schedule,
default_args=default_args)
with dag:
t1 = PythonOperator(
task_id='hello_world',
python_callable=hello_world_py
)
globals()[dag_id] = dag
But this did not worked. I also tried by changing dags_folder value in airflow.cfg to my django views.py file
I also check airflow stable rest api but its doe snot support to create a new dag.
Can anyone please guide on how can i create dags in my Django views.py when my api is being called