airflow and kubernetes deployment

Viewed 21

Airflow version: 2.3.3

I have kubernetes deployment:

kind:Deployment

It pulls container image with python application.

I know how to deploy it into k8s. I know how to schedule it with list cronjob. I know how to create Airflow PythonOperator to build DAG of python tasks.

Question: how to create Airflow DAG to schedule kubernetes deployment using yaml provided so I so not have to migrate all python app code into airflow DAG tasks but trigger one task from airflow which will submit yaml deployment pulling container image into k8s? I to this kind of thing with spark job so I have SparkKubernetesOperator which submit job through yaml. I have KubernetesSensornas well which monitor completion.

In short: I want to schedule this as deployment from Airflow instead of running it as kubernetes cronjob. How to do this?

0 Answers
Related