Kedro install fail to install, but few attempt later it is successful

user_14044307 0
Viewed 594

I have to test if my kedro project works from github so I create a new environment, then :

git clone <my_project>
pip install kedro kedro[pandas] kedro-viz jupyter
kedro build-reqs
kedro install

and the install fails, then I retry a few time (sometimes 2 or 3) then the next attempt it is successful

see image

EDIT: python -V : Python 3.7.10 kedro --version : kedro, version 0.17.3

i cant post my requirement.txt (post is mostly code) so here is my requirement.in

black==v19.10b0
flake8>=3.7.9, <4.0
ipython==7.10
isort>=4.3.21, <5.0
jupyter~=1.0
jupyter_client>=5.1, <7.0
jupyterlab==0.31.1
kedro==0.17.3
nbstripout==0.3.3
pytest-cov~=2.5
pytest-mock>=1.7.1, <2.0
pytest~=6.1.2
wheel==0.32.2
spacy>=3.0.0,<4.0.0
scikit-learn == 0.24.2
kedro-viz==3.11.0
wordcloud== 1.8.1
https://github.com/explosion/spacy-models/releases/download/fr_core_news_sm-3.0.0/fr_core_news_sm-3.0.0.tar.gz#egg=fr_core_news_sm
kedro pipeline python
1 Answers

As indicated in the comment, I think there are two issues at play.

1. Decoding error

This is the main exception you're getting, i.e.:

UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xe8 in position 69: invalid continuation byte

This is unexpectedly raised while Kedro itself is handling the errors from pip install (see this line of Kedro's source code). I believe the cause might be that you have accented characters in your working directory, which can't be interpreted by Python's standard decode() (see this). Example:

b'accélération'.decode()
>> SyntaxError: bytes can only contain ASCII literal characters.

The decoding error is obscuring the actual pip install error.

2. pip install error

As you correctly pointed out, kedro install uses pip install under the hood. It's a bit difficult to pinpoint the exact cause without seeing the actual error. I could however reproduce a similar issue, in my case getting the following error:

ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'c:\\users\\<mu-user>\\anaconda3\\<my-env>\\kedro_project_tests\\lib\\site-packages\\~ydantic\\annotated_types.cp37-win_amd64.pyd'
Consider using the `--user` option or check the permissions.

I believe this is caused by interactions caused by between different versions Kedro and Kedro-Viz. Simply not pip installing kedro-viz before doing kedro install fixed it for me.

Note: Related to this, there will surely be an error if the version of Kedro installed through pip before doing kedro install is not the same as the version of Kedro specified in requirements.in or requirements.txt. This is obvious, as the package currently handling execution will be uninstalled. The error in this case will be something like this:

ERROR: Could not install packages due to an OSError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\<my-user>\\anaconda3\\envs\\<my-env>\\scripts\\kedro.exe
user_89298550
Related