where to write DAG files in apache air flow?

user_13503658 0
Viewed 34

I just started learning apache airflow, and I created an environment in composer in gcp and web server is working fine and everything, but I was just confused about where to write the DAG file? I mean I want to write the file where I can test it multiple times because in the web UI it's showing me a bucket where I can store the file, but I am unable to understand where to write the code. do I have to install airflow in my machine?

p.s - i know this is a stupid question, any help will be appreciated

airflow google-cloud-platform
1 Answers

You can install it locally yes. If you want to test it locally this is the only way I think.

There are couple of tools that you could do that - there is an astro CLI for managing your "dag development" environment which is published by Astronomer, https://github.com/astronomer/astro-cli

Also MWAA has their own tool too - I think, I think Composer has no Composer-specific one.

However for "generic" Airflow (which should be enough to start), you can use the community managed quick-start (either with local venv or Docker-Compose):

https://airflow.apache.org/docs/apache-airflow/stable/start/index.html

user_5167010
Related