Cloud Composer, Airflow - web_server_worker_timeout environmental variable

user_1157751 0
Viewed 352

I need to get accounts from a web service that might take over the current default of 60 seconds.

It seems that my workers are constantly going into timeout, which can be modified under the web_server_worker_timeout environmental variable in webserver group. However, this variable cannot be modified under cloud composer.

Any way to get around this?

google-cloud-composer airflow
1 Answers

There is no way to custom this, but in composer2 you can select the kubernetes limit cpu for the webserver.

with 0.5cpu on a ENVIRONMENT_SIZE_SMALL my webserver usually fail during 5 minutes ( the 2 workers timeout ) before reaching a stable state.

Try setting more than 0.5cpu to your webserver

user_62275000
Related