how to solve cpp library confliction within anaconda?

Lei Yu 0
Viewed 28

I tried to install lightgbm with gpu in anaconda. I used pip in anaconda directly with --install-option='--gpu'. it's built successfully and lib_lightgbm.so links to libstdc++.so under /lib64.

as anaconda has it's own libstdc++.so under anaconda3/lib and it's different from the one under /lib64, when I try to import lightgbm, I got the error saying

anaconda3/lib/libstdc++.so.6: version GLIBCXX_3.4.29 not found (required by lib_lightgbm.so)

what's the recommend way to keep libstdc++.so in anaconda consistent with lightgbm (or any other libraries) built outside anaconda?

do I need to find the cpp compiler information used by anaconda? where can I find such information?

lightgbm anaconda3 anaconda c++ python
1 Answers

As described in microsoft/LightGBM#5106, when building lightgbm from source in a conda environment on Linux, the most reliable way to avoid linking to a libstdc++ that is using a new GLIBCXX version than the one found by conda is to use conda's C/C++ compilers.

Like the following.

# create conda env with lightgbm's dependencies
conda create \
   --name lightgbm-env \
   --yes \
   -c conda-forge \
   python=3.10 \
   numpy \
   scikit-learn \
   scipy

# install `conda`'s C++ build tools into that environment
conda install \
   --name lightgbm-env \
   --yes \
   -c conda-forge \
   python=3.10 \
   cmake \
   gcc_linux-64 \
   gxx_linux-64

# clone LightGBM
git clone \
   --recursive \
   --branch stable \
   https://github.com/microsoft/LightGBM.git

# activate the conda env, so that conda's compilers are found first
source activate lightgbm-env

# install lightgbm with `pip`
cd LightGBM/python-package
pip install .

NOTE: This answer applies to lightgbm versions <=3.3.2.99.

James Lamb0
Related