I'm trying to upgrade Airflow to the Latest version by following the steps here - https://airflow.apache.org/docs/apache-airflow/stable/upgrading-from-1-10/index.html
Everything went well untill I executed this command -
pip install -U apache-airflow
Result of above command :
Requirement already satisfied: apache-airflow in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (1.10.15)
Collecting apache-airflow
Downloading apache_airflow-2.2.3-py3-none-any.whl (5.3 MB)
|████████████████████████████████| 5.3 MB 22.5 MB/s
Requirement already satisfied: cattrs<1.1.0,>=1.0 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (1.0.0)
Requirement already satisfied: attrs<21.0,>=20.0 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (20.3.0)
Requirement already satisfied: markdown<4.0,>=2.5.2 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (2.6.11)
Collecting cryptography>=0.9.3
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
Collecting importlib-resources~=5.2
Downloading importlib_resources-5.4.0-py3-none-any.whl (28 kB)
Requirement already satisfied: lockfile>=0.12.2 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.12.2)
Collecting dataclasses
Downloading dataclasses-0.8-py3-none-any.whl (19 kB)
Collecting tenacity>=6.2.0
Downloading tenacity-8.0.1-py3-none-any.whl (24 kB)
Requirement already satisfied: setproctitle<2,>=1.1.8 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (1.1.10)
Collecting httpx<0.20.0
Downloading httpx-0.19.0-py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB 8.1 MB/s
Requirement already satisfied: importlib-metadata>=1.7 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (2.1.3)
Collecting apache-airflow-providers-ftp
Downloading apache_airflow_providers_ftp-2.0.1-py3-none-any.whl (15 kB)
Collecting python3-openid~=3.2
Downloading python3_openid-3.2.0-py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 68.9 MB/s
Requirement already satisfied: flask<2.0,>=1.1.0 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (1.1.1)
Requirement already satisfied: flask-login<0.5,>=0.3 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (0.4.1)
Collecting inflection>=0.3.1
Downloading inflection-0.5.1-py2.py3-none-any.whl (9.5 kB)
Collecting marshmallow-oneofschema>=2.0.1
Downloading marshmallow_oneofschema-3.0.1-py2.py3-none-any.whl (5.8 kB)
Collecting flask-caching<2.0.0,>=1.5.0
Downloading Flask_Caching-1.10.1-py3-none-any.whl (34 kB)
Collecting python-daemon>=2.2.4
Downloading python_daemon-2.3.0-py2.py3-none-any.whl (35 kB)
Requirement already satisfied: graphviz>=0.12 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.13.2)
Requirement already satisfied: termcolor>=1.1.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (1.1.0)
Requirement already satisfied: jsonschema~=3.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (3.1.1)
Requirement already satisfied: python-nvd3~=0.15.0 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (0.15.0)
Collecting sqlalchemy-jsonfield~=1.0
Downloading SQLAlchemy_JSONField-1.0.0-py3-none-any.whl (10 kB)
Requirement already satisfied: markupsafe>=1.1.1 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (1.1.1)
Collecting werkzeug>=1.0.1,~=1.0
Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
|████████████████████████████████| 298 kB 50.4 MB/s
Requirement already satisfied: python-slugify<5.0,>=3.0.0 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (4.0.1)
Requirement already satisfied: tabulate<0.9,>=0.7.5 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.8.6)
Collecting apache-airflow-providers-http
Downloading apache_airflow_providers_http-2.0.3-py3-none-any.whl (21 kB)
Collecting sqlalchemy>=1.3.18
Downloading SQLAlchemy-1.4.31-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
|████████████████████████████████| 1.6 MB 73.3 MB/s
Requirement already satisfied: unicodecsv>=0.14.1 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.14.1)
Collecting apache-airflow-providers-sqlite
Downloading apache_airflow_providers_sqlite-2.1.0-py3-none-any.whl (15 kB)
Requirement already satisfied: packaging>=14.0 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (21.3)
Collecting pendulum~=2.0
Downloading pendulum-2.1.2-cp36-cp36m-manylinux1_x86_64.whl (154 kB)
|████████████████████████████████| 154 kB 79.3 MB/s
Requirement already satisfied: cached-property~=1.5 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (1.5.1)
Requirement already satisfied: croniter<1.1,>=0.3.17 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.3.30)
Requirement already satisfied: lazy-object-proxy in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (1.4.3)
Requirement already satisfied: jinja2<4,>=2.10.1 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (2.10.3)
Collecting clickclick>=1.2
Downloading clickclick-20.10.2-py2.py3-none-any.whl (7.4 kB)
Collecting apache-airflow-providers-imap
Downloading apache_airflow_providers_imap-2.2.0-py3-none-any.whl (16 kB)
Requirement already satisfied: typing-extensions>=3.7.4 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (4.1.1)
Requirement already satisfied: colorlog<6.0,>=4.0.2 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (4.0.2)
Requirement already satisfied: wtforms<3.0.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (2.2.1)
Collecting swagger-ui-bundle>=0.0.2
Downloading swagger_ui_bundle-0.0.9-py3-none-any.whl (6.2 MB)
|████████████████████████████████| 6.2 MB 52.5 MB/s
Requirement already satisfied: itsdangerous<2.0,>=1.1.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (1.1.0)
Requirement already satisfied: psutil<6.0.0,>=4.2.0 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (5.6.5)
Requirement already satisfied: pygments<3.0,>=2.0.1 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (2.4.2)
Requirement already satisfied: argcomplete~=1.10 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (1.10.2)
Requirement already satisfied: docutils<0.17 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.15.2)
Collecting alembic<2.0,>=1.5.1
Downloading alembic-1.7.6-py3-none-any.whl (210 kB)
|████████████████████████████████| 210 kB 55.3 MB/s
Requirement already satisfied: pep562~=1.0 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from apache-airflow) (1.1)
Collecting flask-appbuilder<4.0.0,>=3.3.2
Downloading Flask_AppBuilder-3.4.4-py3-none-any.whl (1.9 MB)
|████████████████████████████████| 1.9 MB 67.7 MB/s
Collecting flask-wtf<0.15,>=0.14.3
Downloading Flask_WTF-0.14.3-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: iso8601>=0.1.12 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.1.12)
Collecting openapi-spec-validator>=0.2.4
Downloading openapi_spec_validator-0.3.3-py2.py3-none-any.whl (31 kB)
Collecting gunicorn>=20.1.0
Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB)
|████████████████████████████████| 79 kB 4.1 MB/s
Requirement already satisfied: python-dateutil<3,>=2.3 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (2.8.1)
Collecting rich>=9.2.0
Downloading rich-11.2.0-py3-none-any.whl (217 kB)
|████████████████████████████████| 217 kB 57.9 MB/s
Requirement already satisfied: dill<0.4,>=0.2.2 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (0.3.1.1)
Collecting blinker
Downloading blinker-1.4.tar.gz (111 kB)
|████████████████████████████████| 111 kB 54.1 MB/s
Preparing metadata (setup.py) ... done
Requirement already satisfied: pyyaml>=5.1 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from apache-airflow) (5.1.2)
Requirement already satisfied: pyjwt<2 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from apache-airflow) (1.7.1)
Requirement already satisfied: Mako in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from alembic<2.0,>=1.5.1->apache-airflow) (1.1.0)
Requirement already satisfied: click>=4.0 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from clickclick>=1.2->apache-airflow) (7.0)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (405 kB)
Collecting apispec[yaml]<4,>=3.3
Downloading apispec-3.3.2-py2.py3-none-any.whl (27 kB)
Collecting marshmallow<4,>=3
Downloading marshmallow-3.14.1-py3-none-any.whl (47 kB)
|████████████████████████████████| 47 kB 8.6 MB/s
Requirement already satisfied: prison<1.0.0,>=0.2.1 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (0.2.1)
Requirement already satisfied: Flask-SQLAlchemy<3,>=2.4 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (2.4.1)
Requirement already satisfied: marshmallow-enum<2,>=1.5.1 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (1.5.1)
Requirement already satisfied: sqlalchemy-utils<1,>=0.32.21 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (0.38.2)
Requirement already satisfied: email-validator<2,>=1.0.5 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (1.1.3)
Requirement already satisfied: Flask-OpenID<2,>=1.2.5 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (1.2.5)
Requirement already satisfied: Flask-JWT-Extended<4,>=3.18 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (3.24.1)
Collecting marshmallow-sqlalchemy<0.27.0,>=0.22.0
Downloading marshmallow_sqlalchemy-0.26.1-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: colorama<1,>=0.3.9 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (0.4.1)
Requirement already satisfied: Flask-Babel<3,>=1 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (1.0.0)
Requirement already satisfied: setuptools>=3.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from gunicorn>=20.1.0->apache-airflow) (41.6.0)
Collecting charset-normalizer
Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting async-generator
Downloading async_generator-1.10-py3-none-any.whl (18 kB)
Collecting sniffio
Downloading sniffio-1.2.0-py3-none-any.whl (10 kB)
Collecting httpcore<0.14.0,>=0.13.3
Downloading httpcore-0.13.7-py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 11.1 MB/s
Requirement already satisfied: certifi in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from httpx<0.20.0->apache-airflow) (2019.9.11)
Collecting rfc3986[idna2008]<2,>=1.3
Downloading rfc3986-1.5.0-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: zipp>=0.5 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from importlib-metadata>=1.7->apache-airflow) (0.6.0)
Collecting zipp>=0.5
Downloading zipp-3.6.0-py3-none-any.whl (5.3 kB)
Requirement already satisfied: pyrsistent>=0.14.0 in /home/ec2-user/softwares/etl/lib64/python3.6/site-packages (from jsonschema~=3.0->apache-airflow) (0.15.5)
Requirement already satisfied: six>=1.11.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from jsonschema~=3.0->apache-airflow) (1.13.0)
Collecting jsonschema~=3.0
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
|████████████████████████████████| 56 kB 8.5 MB/s
Collecting openapi-schema-validator<0.2.0
Downloading openapi_schema_validator-0.1.6-py3-none-any.whl (7.9 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from packaging>=14.0->apache-airflow) (3.0.7)
Collecting pytzdata>=2020.1
Downloading pytzdata-2020.1-py2.py3-none-any.whl (489 kB)
|████████████████████████████████| 489 kB 66.7 MB/s
Requirement already satisfied: text-unidecode>=1.3 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from python-slugify<5.0,>=3.0.0->apache-airflow) (1.3)
Requirement already satisfied: defusedxml in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from python3-openid~=3.2->apache-airflow) (0.6.0)
Collecting commonmark<0.10.0,>=0.9.0
Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
|████████████████████████████████| 51 kB 10.1 MB/s
Collecting pygments<3.0,>=2.0.1
Downloading Pygments-2.11.2-py3-none-any.whl (1.1 MB)
|████████████████████████████████| 1.1 MB 13.3 MB/s
Collecting greenlet!=0.4.17
Downloading greenlet-1.1.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (147 kB)
|████████████████████████████████| 147 kB 45.7 MB/s
Requirement already satisfied: typing>=3.6 in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from sqlalchemy-jsonfield~=1.0->apache-airflow) (3.7.4.3)
Collecting requests>=2.26.0
Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
|████████████████████████████████| 63 kB 2.8 MB/s
Requirement already satisfied: pycparser in /home/ec2-user/softwares/etl/lib/python3.6/dist-packages (from cffi>=1.12->cryptography>=0.9.3->apache-airflow) (2.21)
Requirement already satisfied: idna>=2.0.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from email-validator<2,>=1.0.5->flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (2.8)
Requirement already satisfied: dnspython>=1.15.0 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from email-validator<2,>=1.0.5->flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (1.16.0)
Requirement already satisfied: pytz in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from Flask-Babel<3,>=1->flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (2019.3)
Requirement already satisfied: Babel>=2.3 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from Flask-Babel<3,>=1->flask-appbuilder<4.0.0,>=3.3.2->apache-airflow) (2.7.0)
Collecting anyio==3.*
Downloading anyio-3.5.0-py3-none-any.whl (79 kB)
|████████████████████████████████| 79 kB 10.7 MB/s
Collecting h11<0.13,>=0.11
Downloading h11-0.12.0-py3-none-any.whl (54 kB)
|████████████████████████████████| 54 kB 4.4 MB/s
Collecting contextvars>=2.1
Downloading contextvars-2.4.tar.gz (9.6 kB)
Preparing metadata (setup.py) ... done
Collecting isodate
Downloading isodate-0.6.1-py2.py3-none-any.whl (41 kB)
|████████████████████████████████| 41 kB 892 kB/s
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/softwares/etl/lib/python3.6/site-packages (from requests>=2.26.0->apache-airflow-providers-http->apache-airflow) (1.25.7)
Collecting immutables>=0.9
Downloading immutables-0.16-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (103 kB)
|████████████████████████████████| 103 kB 75.1 MB/s
Building wheels for collected packages: blinker, contextvars
Building wheel for blinker (setup.py) ... done
Created wheel for blinker: filename=blinker-1.4-py3-none-any.whl size=13451 sha256=fcf9e396978c821a6f95c3c4bb3fb02ae0f30a36d10c83b61918f46a55746ec9
Stored in directory: /home/ec2-user/.cache/pip/wheels/4f/4a/93/c5ed8c11fedbe97fb8b8032b301eaa736248684b44087a7259
Building wheel for contextvars (setup.py) ... done
Created wheel for contextvars: filename=contextvars-2.4-py3-none-any.whl size=7665 sha256=ce680f0367b50496d8c842571ca00663a14b554010617caea24cd1b2175e7dc6
Stored in directory: /home/ec2-user/.cache/pip/wheels/41/11/53/911724983aa48deb94792432e14e518447212dd6c5477d49d3
Successfully built blinker contextvars
Installing collected packages: immutables, zipp, contextvars, werkzeug, sniffio, greenlet, dataclasses, sqlalchemy, rfc3986, python3-openid, marshmallow, jsonschema, isodate, h11, charset-normalizer, apispec, anyio, requests, pytzdata, pygments, openapi-schema-validator, marshmallow-sqlalchemy, importlib-resources, httpcore, flask-wtf, commonmark, cffi, async-generator, tenacity, swagger-ui-bundle, sqlalchemy-jsonfield, rich, python-daemon, pendulum, openapi-spec-validator, marshmallow-oneofschema, inflection, httpx, gunicorn, flask-caching, flask-appbuilder, cryptography, clickclick, blinker, apache-airflow-providers-sqlite, apache-airflow-providers-imap, apache-airflow-providers-http, apache-airflow-providers-ftp, alembic, apache-airflow
WARNING: Value for scheme.purelib does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/ec2-user/softwares/etl/lib/python3.6/dist-packages
sysconfig: /home/ec2-user/softwares/etl/lib/python3.6/site-packages
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Attempting uninstall: zipp
Found existing installation: zipp 0.6.0
Uninstalling zipp-0.6.0:
Successfully uninstalled zipp-0.6.0
Attempting uninstall: werkzeug
Found existing installation: Werkzeug 0.16.0
Uninstalling Werkzeug-0.16.0:
Successfully uninstalled Werkzeug-0.16.0
Attempting uninstall: sqlalchemy
Found existing installation: SQLAlchemy 1.3.11
Uninstalling SQLAlchemy-1.3.11:
Successfully uninstalled SQLAlchemy-1.3.11
Attempting uninstall: python3-openid
Found existing installation: python3-openid 3.1.0
Uninstalling python3-openid-3.1.0:
Successfully uninstalled python3-openid-3.1.0
Attempting uninstall: marshmallow
Found existing installation: marshmallow 2.19.5
Uninstalling marshmallow-2.19.5:
Successfully uninstalled marshmallow-2.19.5
Attempting uninstall: jsonschema
Found existing installation: jsonschema 3.1.1
Uninstalling jsonschema-3.1.1:
Successfully uninstalled jsonschema-3.1.1
Attempting uninstall: apispec
Found existing installation: apispec 1.3.3
Uninstalling apispec-1.3.3:
Successfully uninstalled apispec-1.3.3
Attempting uninstall: requests
Found existing installation: requests 2.22.0
Uninstalling requests-2.22.0:
Successfully uninstalled requests-2.22.0
Attempting uninstall: pytzdata
Found existing installation: pytzdata 2019.3
Uninstalling pytzdata-2019.3:
Successfully uninstalled pytzdata-2019.3
Attempting uninstall: pygments
Found existing installation: Pygments 2.4.2
Uninstalling Pygments-2.4.2:
Successfully uninstalled Pygments-2.4.2
Attempting uninstall: marshmallow-sqlalchemy
Found existing installation: marshmallow-sqlalchemy 0.18.0
Uninstalling marshmallow-sqlalchemy-0.18.0:
Successfully uninstalled marshmallow-sqlalchemy-0.18.0
Attempting uninstall: importlib-resources
Found existing installation: importlib-resources 1.5.0
Uninstalling importlib-resources-1.5.0:
Successfully uninstalled importlib-resources-1.5.0
Attempting uninstall: flask-wtf
Found existing installation: Flask-WTF 0.14.2
Uninstalling Flask-WTF-0.14.2:
Successfully uninstalled Flask-WTF-0.14.2
Attempting uninstall: tenacity
Found existing installation: tenacity 4.12.0
Uninstalling tenacity-4.12.0:
Successfully uninstalled tenacity-4.12.0
Attempting uninstall: sqlalchemy-jsonfield
Found existing installation: SQLAlchemy-JSONField 0.9.0
Uninstalling SQLAlchemy-JSONField-0.9.0:
Successfully uninstalled SQLAlchemy-JSONField-0.9.0
Attempting uninstall: python-daemon
Found existing installation: python-daemon 2.1.2
Uninstalling python-daemon-2.1.2:
Successfully uninstalled python-daemon-2.1.2
Attempting uninstall: pendulum
Found existing installation: pendulum 1.4.4
Uninstalling pendulum-1.4.4:
Successfully uninstalled pendulum-1.4.4
Attempting uninstall: gunicorn
Found existing installation: gunicorn 19.9.0
Uninstalling gunicorn-19.9.0:
Successfully uninstalled gunicorn-19.9.0
Attempting uninstall: flask-caching
Found existing installation: Flask-Caching 1.3.3
Uninstalling Flask-Caching-1.3.3:
Successfully uninstalled Flask-Caching-1.3.3
Attempting uninstall: flask-appbuilder
Found existing installation: Flask-AppBuilder 2.3.4
Uninstalling Flask-AppBuilder-2.3.4:
Successfully uninstalled Flask-AppBuilder-2.3.4
Attempting uninstall: alembic
Found existing installation: alembic 1.3.1
Uninstalling alembic-1.3.1:
Successfully uninstalled alembic-1.3.1
Attempting uninstall: apache-airflow
Found existing installation: apache-airflow 1.10.15
Uninstalling apache-airflow-1.10.15:
Successfully uninstalled apache-airflow-1.10.15
Successfully installed alembic-1.7.6 anyio-3.5.0 apache-airflow-2.2.3 apache-airflow-providers-ftp-2.0.1 apache-airflow-providers-http-2.0.3 apache-airflow-providers-imap-2.2.0 apache-airflow-providers-sqlite-2.1.0 apispec-3.3.2 async-generator-1.10 blinker-1.4 cffi-1.15.0 charset-normalizer-2.0.12 clickclick-20.10.2 commonmark-0.9.1 contextvars-2.4 cryptography-36.0.1 dataclasses-0.8 flask-appbuilder-3.4.4 flask-caching-1.10.1 flask-wtf-0.14.3 greenlet-1.1.2 gunicorn-20.1.0 h11-0.12.0 httpcore-0.13.7 httpx-0.19.0 immutables-0.16 importlib-resources-5.4.0 inflection-0.5.1 isodate-0.6.1 jsonschema-3.2.0 marshmallow-3.14.1 marshmallow-oneofschema-3.0.1 marshmallow-sqlalchemy-0.26.1 openapi-schema-validator-0.1.6 openapi-spec-validator-0.3.3 pendulum-2.1.2 pygments-2.11.2 python-daemon-2.3.0 python3-openid-3.2.0 pytzdata-2020.1 requests-2.27.1 rfc3986-1.5.0 rich-11.2.0 sniffio-1.2.0 sqlalchemy-1.4.31 sqlalchemy-jsonfield-1.0.0 swagger-ui-bundle-0.0.9 tenacity-8.0.1 werkzeug-1.0.1 zipp-3.6.0
After executing the above command, I wanted to check the Airflow version and I got this error.
ModuleNotFoundError: No module named 'pendulum'
So, I ran this command -
pip install pendulum
To my surprise, I got even more errors
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
apache-airflow 2.2.3 requires cryptography>=0.9.3, which is not installed.
apache-airflow 2.2.3 requires markupsafe>=1.1.1, which is not installed.
apache-airflow 2.2.3 requires psutil<6.0.0,>=4.2.0, which is not installed.
apache-airflow 2.2.3 requires pyyaml>=5.1, which is not installed.
apache-airflow 2.2.3 requires setproctitle<2,>=1.1.8, which is not installed.
apache-airflow 2.2.3 requires sqlalchemy>=1.3.18, which is not installed.
Successfully installed pendulum-2.1.2
It says that pendulum-2.1.2 is Installed, but the same error(No module named 'pendulum') appears when I check the airflow version
Any suggestions?
Details :
Airflow version upgrade from 1.10.15 to 2.2.3(latest version)
Machine used - Amazon EC2 instance
Python version - 3.6.8
pip version - 21.3.1