I have an Airflow DAG that scrapes some data from an external website. The site authenticates first with username and password, then challenges with a 2fa token. Is there a way to have an Airflow DAG pause in the middle and wait for user input for the 2fa token value?
I've thought that maybe using a Sensor might work, either a Task or File Sensor, but I haven't been able to get that to work so far (sorry, new to Airflow!). It feels like that should work in theory, but I'm not sure if there is an easier approach.
Thanks!