C# linker unable to find python shared library on Ubuntu 18.04 on Keras.NET program

Viewed 527

I am fairly new to using C# (Come from a c/c++ background) and have been trying to develop a C# machine learning application using Keras.NET (https://github.com/reneschulte/Keras.NET). Using the terminal provided in VS Code, I added the Keras.NET package to my .csproj file using the dotnet add package commands along with the packages that the Keras.NET's webpage says it uses (ie Numpy.NET (using Numpy.Bare), pythonnet_netstandard, and Tensorflow.NET). I have also installed python 3.7.3.

I am using Ubuntu 18.04 as my OS and VS Code as my IDE. I have been trying get an simple neural network using the MNIST dataset to work but have run into a run-time issue when using dotnet run :

Unhandled Exception: System.DllNotFoundException: Unable to load shared library 'python37' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libpython37: cannot open shared object file: No such file or directory

I dug around my computer to find out where my python 3.7 libraries were and found what I think the linker is looking for here in ~/usr/local/lib (and there were no python 3.7.so files in ~/usr/lib).

Location of Python 3.7 shared objects

So to ensure the linker was looking in the right places I added the path to the linker library search path using:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/ on my VS Code terminal

Finally I used LD_DEBUG to find out where the linker was searching for the python37 library and this was in the LD_DEBUG output file:

      9035:  search path=tls/haswell/x86_64:tls/haswell:tls/x86_64:tls:haswell/x86_64:haswell:x86_64::/usr/local/lib        (LD_LIBRARY_PATH)
      9035:   trying file=tls/haswell/x86_64/python37.so
      9035:   trying file=tls/haswell/python37.so
      9035:   trying file=tls/x86_64/python37.so
      9035:   trying file=tls/python37.so
      9035:   trying file=haswell/x86_64/python37.so
      9035:   trying file=haswell/python37.so
      9035:   trying file=x86_64/python37.so
      9035:   trying file=python37.so
      9035:   trying file=/usr/local/lib/python37.so
      9035:  search cache=/etc/ld.so.cache
      9035:  search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/tls:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib        (system search path)
      9035:   trying file=/lib/x86_64-linux-gnu/python37.so
      9035:   trying file=/usr/lib/x86_64-linux-gnu/tls/python37.so
      9035:   trying file=/usr/lib/x86_64-linux-gnu/python37.so
      9035:   trying file=/lib/python37.so
      9035:   trying file=/usr/lib/python37.so
      9035: 
      9035: find library=libpython37.so [0]; searching
      9035:  search path=tls/haswell/x86_64:tls/haswell:tls/x86_64:tls:haswell/x86_64:haswell:x86_64::/usr/local/lib        (LD_LIBRARY_PATH)
      9035:   trying file=tls/haswell/x86_64/libpython37.so
      9035:   trying file=tls/haswell/libpython37.so
      9035:   trying file=tls/x86_64/libpython37.so
      9035:   trying file=tls/libpython37.so
      9035:   trying file=haswell/x86_64/libpython37.so
      9035:   trying file=haswell/libpython37.so
      9035:   trying file=x86_64/libpython37.so
      9035:   trying file=libpython37.so
      9035:   trying file=/usr/local/lib/libpython37.so
      9035:  search cache=/etc/ld.so.cache
      9035:  search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/tls:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib        (system search path)
      9035:   trying file=/lib/x86_64-linux-gnu/libpython37.so
      9035:   trying file=/usr/lib/x86_64-linux-gnu/tls/libpython37.so
      9035:   trying file=/usr/lib/x86_64-linux-gnu/libpython37.so
      9035:   trying file=/lib/libpython37.so
      9035:   trying file=/usr/lib/libpython37.so
      9035: 
      9035: find library=python37 [0]; searching
      9035:  search path=tls/haswell/x86_64:tls/haswell:tls/x86_64:tls:haswell/x86_64:haswell:x86_64::/usr/local/lib        (LD_LIBRARY_PATH)
      9035:   trying file=tls/haswell/x86_64/python37
      9035:   trying file=tls/haswell/python37
      9035:   trying file=tls/x86_64/python37
      9035:   trying file=tls/python37
      9035:   trying file=haswell/x86_64/python37
      9035:   trying file=haswell/python37
      9035:   trying file=x86_64/python37
      9035:   trying file=python37
      9035:   trying file=/usr/local/lib/python37
      9035:  search cache=/etc/ld.so.cache
      9035:  search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/tls:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib        (system search path)
      9035:   trying file=/lib/x86_64-linux-gnu/python37
      9035:   trying file=/usr/lib/x86_64-linux-gnu/tls/python37
      9035:   trying file=/usr/lib/x86_64-linux-gnu/python37
      9035:   trying file=/lib/python37
      9035:   trying file=/usr/lib/python37
      9035: 
      9035: find library=libpython37 [0]; searching
      9035:  search path=tls/haswell/x86_64:tls/haswell:tls/x86_64:tls:haswell/x86_64:haswell:x86_64::/usr/local/lib        (LD_LIBRARY_PATH)
      9035:   trying file=tls/haswell/x86_64/libpython37
      9035:   trying file=tls/haswell/libpython37
      9035:   trying file=tls/x86_64/libpython37
      9035:   trying file=tls/libpython37
      9035:   trying file=haswell/x86_64/libpython37
      9035:   trying file=haswell/libpython37
      9035:   trying file=x86_64/libpython37
      9035:   trying file=libpython37
      9035:   trying file=/usr/local/lib/libpython37
      9035:  search cache=/etc/ld.so.cache
      9035:  search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/tls:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib        (system search path)
      9035:   trying file=/lib/x86_64-linux-gnu/libpython37
      9035:   trying file=/usr/lib/x86_64-linux-gnu/tls/libpython37
      9035:   trying file=/usr/lib/x86_64-linux-gnu/libpython37
      9035:   trying file=/lib/libpython37
      9035:   trying file=/usr/lib/libpython37  

Any idea on having the linker find the right shared library?

0 Answers
Related