I run docker-compose up airflow-init in cmd to deploy Airflow using the template given at docker-compose.yaml, folder logs, plugins and dags are created:
C:\Users\Diego\Airflow>dir
Volume in drive C has no label.
Volume Serial Number is 4296-D163
Directory of C:\Users\Diego\Airflow
09/18/2021 07:45 PM <DIR> .
09/18/2021 07:45 PM <DIR> ..
09/18/2021 07:45 PM 44 .env
09/18/2021 05:59 PM <DIR> dags
09/18/2021 05:52 PM 8,031 docker-compose.yaml
09/18/2021 07:44 PM <DIR> logs
09/18/2021 06:02 PM <DIR> plugins
2 File(s) 8,075 bytes
5 Dir(s) 22,549,196,800 bytes free
When the command is run, the Error given is:
C:\Users\Diego\Airflow>docker-compose up airflow-init
WARNING: The AIRFLOW_UID variable is not set. Defaulting to a blank string.
WARNING: The AIRFLOW_GID variable is not set. Defaulting to a blank string.
Creating network "airflow_default" with the default driver
Creating volume "airflow_postgres-db-volume" with default driver
Creating airflow_redis_1 ... done Creating airflow_postgres_1 ... done Creating airflow_airflow-init_1 ... done Attaching to airflow_airflow-init_1
airflow-init_1 | ERROR!!!: AIRFLOW_UID not set!
airflow-init_1 | Please follow these instructions to set AIRFLOW_UID and AIRFLOW_GID environment variables:
airflow-init_1 | https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#initializing-environment
It seems to be an error while synchronizing access from docker to the folders, which is solved in Linux or mac by defining env variables file with echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env .
I tried to run the echo command, which leads to the .env file created in the directory, but this doesn't help to get airflow-init to run without the error, so this is only for Linux, the mounted volumes in container seems unable to use the windows filesystem user/group permissions, so the container and host computer don't have matching file permissions.
How to set up correctly environment variables for airflow using docker-compose in Window 10?
Docker Desktop 4.0.1 Steps:
- Get the file from yaml Airflow community webpage and place it in a directory
- Create folders folder logs, dags, plugins in same directory
- Run
docker-compose up airflow-init