How to replace libm when using ctypes?

user_4859041 0
Viewed 362

I'm getting different results for the same shared library when a function (from the shared library) is executed by a standalone executable and when using Python's ctypes for calling the exact same function.
The difference was narrowed down to be the version of libm used by the library.
When using the library from a standalone executable we link against openlibm, but when the library is linked by ctypes, it's linked against Ubuntu's libm. How can I make ctypes link my shared library against openlibm?

The shared library header (fft.h):

extern "C" {
void fft(const char* srcPath);
};

The shared library implementation (fft.cpp):

#include <iostream>
#include <fstream>
#include <nlohmann/json.hpp>
#include <Eigen/Eigen>
#include <unsupported/Eigen/FFT>

void fft(const char* srcPath) {
    std::cout.precision(17);

    std::ifstream fs(srcPath, std::ofstream::in);

    // Get length of file
    fs.seekg(0, std::ifstream::end);
    auto length = fs.tellg();
    fs.seekg(0, std::ifstream::beg);

    // Read the file
    auto srcRaw = new char[length];
    fs.read(srcRaw, length);

    // Convert data to vector of doubles
    nlohmann::json j = nlohmann::json::parse(srcRaw);
    std::vector<double> srcDoubles = j;

    int n = 65536;

    Eigen::VectorXd srcEigen(n);
    Eigen::VectorXcd dstEigen(n);
    std::copy(srcDoubles.data(), srcDoubles.data() + srcDoubles.size(), srcEigen.data());

    Eigen::FFT<double> fft;
    fft.fwd(dstEigen, srcEigen);

    std::vector<std::complex<double>> dstTmp(dstEigen.data(), dstEigen.data() + dstEigen.size());

    for (size_t i = 0; i < dstTmp.size(); i++) {
        std::cout << "Result[" << i << "] = " << dstTmp[i].real() << "+i*" << dstTmp[i].imag() << std::endl;
    }

    delete[] srcRaw;
}

Using the library from a standalone executable:

#include <fft.h>

int main(int argc, char** argv) {
    fft("input_data");
}

Using Python's ctypes:

from ctypes import c_char_p, CDLL


class LibCFft(CDLL):
    def __init__(self):
        super(LibCFft, self).__init__("libexample.so")
        self._addfunc('fft', [c_char_p], None)

    def _addfunc(self, name, argtypes, restype):
        attr = getattr(self, name)
        attr.argtypes = argtypes
        attr.restype = restype

        return lambda x: x


class CFft(object):
    def fft(self):
        LibCFft().fft("input_data")


if __name__ == '__main__':
    CFft().fft()

Lib build:

clang++ /opt/eigen-eigen-43d9075b23ef -isystem /opt/spdlog/include -isystem /opt/nlohmann/include -O3 -DNDEBUG -fPIC -std=gnu++14 -o CMakeFiles/example.dir/fft.cpp.o -c /home/user/fft.cpp

clang++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libexample.so -o libbugatone.so CMakeFiles/example.dir/generated/fft.cpp.o -lopenlibm

Executable build:

clang++ -O3 -DNDEBUG -latomic -nodefaultlibs -lopenlibm -lc -lc++ -lgcc -lgcc_s -lstdc++ -latomic -lm  CMakeFiles/eigen_tester.dir/eigen_tester.cpp.o  -o eigen_tester libexample.so -lopenlibm

Thanks in advance for the help.

clang++ ctypes shared-libraries python c++
1 Answers

In the executable link step you use BOTH -lm and -lopenlibm which is probably not what you want.

-nodefaultlibs disables automatic inclusion of all standard libs, so if you simply not include -lm but keep -lopenlibm this should probably do what you want.

You colud update LD_LIBRARY_PATH in your environment, to point to your local library folder first, and keep your libopenlibm.so there and also create a soft link to it: 'ln -s libopenlibm.so libm.so'. In most cases that would cause any shared objects to use this as a substitution to a regular libm.so

user_103053500
Related