pip3 Install delta-lake-reader[aws] fails on MacOS Mojave

user_3600487 0
Viewed 217

I am trying to install delta-lake-reader[aws] on my MacBook Pro with MacOS Mojave and it is failing. I have Python 3.9.1 installed on my laptop.

$ pip3 install delta-lake-reader[aws]


Collecting delta-lake-reader[aws]
  Using cached delta_lake_reader-0.2.2-py3-none-any.whl (7.4 kB)
Collecting pyarrow<3.0.0,>=2.0.0
  Using cached pyarrow-2.0.0.tar.gz (58.9 MB)
  Installing build dependencies ... |

After that it hangs for a while, then fails with a lot of errors, and finally, it gives the following output before ending:

ERROR: Cannot install delta-lake-reader[aws]==0.1.0, delta-lake-reader[aws]==0.1.1, delta-lake-reader[aws]==0.2.1 and delta-lake-reader[aws]==0.2.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    delta-lake-reader[aws] 0.2.2 depends on pyarrow<3.0.0 and >=2.0.0
    delta-lake-reader[aws] 0.2.1 depends on pyarrow<3.0.0 and >=2.0.0
    delta-lake-reader[aws] 0.1.1 depends on pyarrow<3.0.0 and >=2.0.0
    delta-lake-reader[aws] 0.1.0 depends on pyarrow<3.0.0 and >=2.0.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
$

I tried uninstalling pyarrow but it installs 3.0.0 again and fails the same way.

How can I overcome this?

delta-lake pyarrow python-3.x amazon-s3
1 Answers

The problem lays in the explicit dependency, that doesn't include pyarrow 3.0.0, but requires 2.x.x version. And pyarrow 2.x.x is not compatible with Python 3.9, that's why your build fails, but it works with Python 3.8, I was able to install it with pip3.8 install 'delta-lake-reader[aws]==0.2.2'. So you have following choice:

  • Use Python 3.8
  • Download repository, and make change in the pyproject.toml to use pyarrow 3.0.0, and after that use poetry tool to build wheel
  • File issue in the project, and wait until author fixes it.
  • If you don't need additional functionality, then you can use official Delta Lake python package that could be installed with pip install deltalake command
Alex_Ott_186270
Related