Failure updating GCP Composer and Airflow image

user_7885903 0
Viewed 962

When I try to update my Composer environment (on GCP) from the version composer-1.7.2-airflow-1.10.2 to composer-1.12.0-airflow-1.10.10 (or any other available) I always get this error:

UPDATE operation on this environment failed Just now with the following error message:
Failed to update image version.
The Cloud Build image build failed: Build failed; check build logs for details

After looking in the build logs I can only see this error below that doesn't help me so much:

The command '/bin/sh -c bash installer.sh $COMPOSER_PYTHON_VERSION  fail' returned a non-zero code: 1
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1

What could be going on?

google-cloud-composer airflow google-cloud-platform
1 Answers

I found this guide in Composer Docs https://cloud.google.com/composer/docs/how-to/managing/updating that gave me some clue about what was going on.

Then I succeeded after doing this:

  1. ensured that these conditions from the link above were met
  2. became Admin, I was editor (that should have permission), but I became Admin to eliminate permissions doubts
  3. removed all PYPI packages from the environment

Emprically speaking, I think the problem was because of the Pypi packages. Some of them was not compatible with the new composer version and this was broking the build. The logs altought didn't let this clear.

I hope it helps because I struggled to find help with this problem.

user_78859030
Related