getting scikit-learn version warning using yml environment

user_14873150 0
Viewed 324

I want to deploy a machine learning model and have the environment yml file and the model pickle file. When I include scikit-learn=0.23.2 to the dependencies, conda automatically uninstall this scikit-learn version and install scikit-learn-0.24.2 . Therefore, I get the following warning when I load the pickle file.

UserWarning: Trying to unpickle estimator DecisionTreeClassifier from version 0.23.2 when using version 0.24.2. This might lead to breaking code or invalid results. Use at your own risk.

Here is the environment:

name: environment
channels:
  - defaults
dependencies:
  - blas=1.0
  - ca-certificates=2021.5.25
  - certifi=2021.5.30
  - icc_rt=2019.0.0
  - intel-openmp=2021.2.0
  - joblib=0.17.0
  - mkl=2020.2
  - mkl-service=2.3.0
  - mkl_fft=1.3.0
  - mkl_random=1.1.1
  - numpy=1.19.2
  - numpy-base=1.19.2
  - openssl=1.1.1k
  - pandas=1.2.4
  - patsy=0.5.1
  - pickleshare=0.7.5
  - pip=21.1.1
  - pyodbc=4.0.30
  - python=3.7.4
  - python-dateutil
  - pytz=2021.1
  - scipy=1.6.2
  - setuptools=52.0.0
  - six=1.15.0
  - sqlite=3.35.4
  - statsmodels=0.12.0
  - threadpoolctl=2.1.0
  - vc=14.2
  - vs2015_runtime=14.27.29016
  - wheel=0.36.2
  - wincertstore=0.2
  - scikit-learn=0.23.2
  - pip:
    - imblearn==0.0
prefix: C:\Users

And the result of conda env create -f environment.yml is:

Installing collected packages: scikit-learn, imbalanced-learn, imblearn
  Attempting uninstall: scikit-learn
    Found existing installation: scikit-learn 0.23.2
    Uninstalling scikit-learn-0.23.2:
      Successfully uninstalled scikit-learn-0.23.2
Successfully installed imbalanced-learn-0.8.0 imblearn-0.0 scikit-learn-0.24.2

I also tried to install scikit-learn=0.23.2 via pip and I didn't get the warning in my local machine while loading the model. But scikit-learn should not installed via pip in the deployment environment. Do you have any idea?

conda virtualenv python
1 Answers

Explanation

Whatever is in the pip: section of a Conda environment YAML gets installed after the Conda environment is created, and is run with the pip install -U command. The -U gives Pip the permission to upgrade any existing packages if it is necessary to install the specified packages. In this particular case, the version of imblearn must be incompatible with the scikit-learn version you have selected.

Remove imblearn

Technically, you should be using imbalanced-learn not imblearn, as stated in the package description. That also means you don't even need to install from PyPI, since imbalanced-learn is available through Conda Forge.

If you require having scikit-learn=0.23 then you must use imbalanced-learn=0.7. This should be under the regular dependencies, not in the pip: section.

user_5709180
Related