How to Load two different Python Modules whose shared library requirements conflict if set globally

user_982995 0
Viewed 520

I am working on a project where I need to use two python modules together:

After following the instructions to install PyMesh from source, I was able to succesfully import pymesh with my Python 3.8 interpreter (on Ubuntu 18).

I then downloaded and installed PyRep. This initially didn't work, and gave the error:

    ~/.local/lib/python3.8/site-packages/pyrep/backend/sim.py in <module>
      1 from .simConst import *
----> 2 from ._sim_cffi import ffi, lib
      3 import numpy as np
      4 import collections
      5 

ImportError: libcoppeliaSim.so.1: cannot open shared object file: No such file or directory

The installation instructions for PyRep said this is because I need to ensure that PyRep has access to the shared libraries in my local CoppeliaSim installation, and that I should add the following lines to my .bashrc:

export COPPELIASIM_ROOT=EDIT/ME/PATH/TO/COPPELIASIM/INSTALL/DIR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT
export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT

Success! I can import pyrep without issue. However, if I now import pymesh again, I get an error:

~/.local/lib/python3.8/site-packages/pymesh2-0.3-py3.8-linux-x86_64.egg/pymesh/Mesh.py in <module>
      3 import numpy as np
      4 
----> 5 import PyMesh
      6 
      7 class Mesh(object):

ImportError: /home/craig/Programs/CoppeliaSim/libtbb.so: file too short

By globally setting $LD_LIBRARY_PATH at bash login as suggested, I've caused pymesh to search for conflicting shared libraries in the Coppelia-Sim Root folder! (Looking for solutions, I found an article which condems changing $LD_LIBRARY_PATH in .bashrc for exactly this reason)

How do I resolve this dilemna? PyRep asks me to add a directory to the LD_LIBRARY_PATH, but PyMesh breaks if I do so (though I can get each to work in isolation).

Attempted Alternatives

I have learned through researching this problem that you can change where a particular shared library looks for dependencies using the patchelf command. So I tried just manually tweaking where the offending PyRep _sim_cffi library with:

patchelf --set-rpath "/home/craig/Programs/CoppeliaSim" ~/.local/lib/python3.8/site-packages/pyrep/backend/_sim_cffi.cpython-38-x86_64-linux-gnu.so

However, this just leads to transitive dependency errors one level down:

~/.local/lib/python3.8/site-packages/pyrep/backend/sim.py in <module>
      1 from .simConst import *
----> 2 from ._sim_cffi import ffi, lib
      3 import numpy as np
      4 import collections
      5 

ImportError: libQt5SerialPort.so.5: cannot open shared object file: No such file or directory
shared-libraries linux python
2 Answers

I've been reading a bit about pipx on old posts, it could be a potential solution.

I leave you here some info.

You can try using pipx.

pipx will do all the work of setting up separate virtual environments for each application, so that all the applications' dependencies remain separate.

If you use virtualenv yourself, you have to switch environments to run a different application. pipx handles the environment for you, so you don't have to mess with the virtualenvs at all.

From the docs:

pipx is made specifically for application installation, as it adds isolation yet still makes the apps available in your shell: pipx creates an isolated environment for each application and its associated packages.

Source: stackoverflow's @jpyams

Hope this was useful ;)

user_143254070

The first thing I could think of was to create a virtual environment and use that to install your packages. But the problem will still persist, as the issue lies in the shared path between the two packages. Some things you can try:

  • Create a shell script to set the LD_LIBRARY_PATH, do a pip install for PyRep. Then run the script.

    MyScript.sh

     #!/bin/bash
 export COPPELIASIM_ROOT=EDIT/ME/PATH/TO/COPPELIASIM/INSTALL/DIR
 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT
 export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT

 pip3 install -r requirements.txt
 python3 setup.py install --user

    Run the script

     ./MyScript

  • You can also set the environment variable for the command which is run, like below

    MyScript.sh

     pip3 install -r requirements.txt
 python3 setup.py install --user

    Run the script

    $ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:EDIT/ME/PATH/TO/COPPELIASIM/INSTALL/DIR ./MyScript

The above solutions will ensure that the corresponding LD_LIBRARY_PATH is only set when trying to install PyRep and the subsequent path issues for PyMesh shouldn't arise.

user_103577420
Related