I am running Airflow locally inside the Docker containers. It works well and displays log files when I use standard operators, such as PythonOperator, BigQueryExecuteQueryOperator. The logs are created and stored correctly, as well as visible from Airflow UI.
However, whenever I try to use some of the (perfectly working in Cloud Composer) custom operators, no matter what services they connect to, how they work etc., all tasks display the same issue:
*** Log file does not exist: /home/airflow/airflow/logs/linkedin_avaus_daily_load/get_campaign/2022-09-19T09:57:50.869266+00:00/2.log
*** Fetching from: http://:8793/log/linkedin_avaus_daily_load/get_campaign/2022-09-19T09:57:50.869266+00:00/2.log
*** Failed to fetch log file from worker. Request URL missing either an 'http://' or 'https://' protocol.
It is right in saying that "Log file does not exist", as manually checking in the container confirms that. I presume that the task fails somewhere else and Airflow just doesn't manage to store logs. Has anyone else had this issue?
Thank you a lot for all the help.