Airflow: How to unpause DAG from python script

user_10402376 0
Viewed 2002

I'm creating a python script to generate DAGs (generate a new python file with the DAG specifications) from templates. It all works fine, except that I need the DAG to be be generated as unpaused.

I've searched and tried to run shell commands in the script like this: 

bash_command1 = 'airflow list_dags'

bash_command2 = 'airflow trigger_dag ' + str(DAG_ID)

bash_command3 = 'airflow list_tasks ' + str(DAG_ID)

bash_command4 = 'airflow unpause '+ str(DAG_ID)

subprocess.call(bash_command1.split())
subprocess.call(bash_command2.split())
subprocess.call(bash_command3.split())
subprocess.call(bash_command4.split())

But every time I create a new DAG it is shown as paused in the web UI. By the research I´ve made, the command airflow unpause <dag_id> should solve the problem, but when the script executes it, I get the error: 

Traceback (most recent call last):
  File "/home/cubo/anaconda2/bin/airflow", line 28, in <module>
    args.func(args)
  File "/home/cubo/anaconda2/lib/python2.7/site-packages/airflow/bin/cli.py", line 303, in unpause
    set_is_paused(False, args, dag)
  File "/home/cubo/anaconda2/lib/python2.7/site- packages/airflow/bin/cli.py", line 312, in set_is_paused
    dm.is_paused = is_paused
AttributeError: 'NoneType' object has no attribute 'is_paused'

But when I execute the same airflow unpause <dag_id> command in the terminal it works fine, and it prints:

Dag: <DAG: DAG_ID>, paused: False

Any help would be greatly appreciated.

airflow python-2.7
1 Answers

Airflow (1.8 and newer) pauses new dags by default. If you want all DAGs to be unpaused at creation, you can override the Airflow config to retain the prior behavior of unpausing at creation.

Here's the link that walks you through setting configuration options. You want to set a core configuration setting: dags_are_paused_at_creation to False.

We use the environment variable approach on my team.

user_46773650
Related