How can I find out if a DAG is paused/unpaused in Airflow?

user_7129814 0
Viewed 2858

I would like to pause DAGs that are idle and redundant, How do I know which DAGs are unpaused and which are paused?

So I have a list of DAGs that are to be unpaused using a bashcommand that executes airflow pause <dag_id>. I would like to know if the command is successful or not by checking the state of pause of each DAGs. I've checked the airflow webserver and it seems that all of my paused DAGs are still running.

def pause_idle_dags(dags = ["myTutorial"]):
    """
    Pauses dags from the airflow
    :param dags: dags considered to be idle
    :return: Success state
    """
    # TODO
    for dag in dags:
        command = "airflow pause {}".format(dag)
        print(executeBashCommand(command))

def executeBashCommand(command):
    print('========RUN========', command)
    p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdout, stderr = p.communicate()
    if p.returncode != 0:
        print('========STDOUT========\n',stdout.decode())
        print('========STDERR========\n',stderr.decode())
        raise Exception('There is error while executing bash command: '+command+'\nwith log:\n'+stderr.decode())
    return stdout, stderr
airflow-scheduler airflow directed-acyclic-graphs python
1 Answers

When you run airflow commands that instruct it to perform some action it should edit the internal stat of the backend database that it is connected to. By default airflow using SQLite. You may have setup your own one. Either way you can query that to check.

e.g. airflow=# select * from dag where is_paused;

Here you can obviously also perform updates such as

airflow=# update dag set is_paused='false' where is_paused;

user_90251670
Related