Building python 3 c modules in MacOS - setup using odd compiler options

user_10869644 0
Viewed 57

I'm trying to build a c module in MacOS. The build fails during the linker process with the error:

ld: warning: object file (build/temp.macosx-11.1-x86_64-3.7/pparser.o) was built for newer macOS version (11.1) than being linked (11.0)

I realize this is just a warning but the resulting so file fails in import to a python script. I'm doing the build with the following setup.py

from distutils.core import setup, Extension

setup(name='pparser', version='1.0',  \
            ext_modules=[Extension('pparser', 
                                  ['pparser.cpp'],
                         extra_compile_args = ["-Wno-nullability-completeness",
                                               "-Wno-undef-prefix",
                                               "-I/usr/local/opt/flex/include",
                                               "-I/usr/local/opt/bison/include",
                                               "-std=gnu++14"])])

What's odd is that I can capture the clang invocation and run it myself at the command line with the verbose option

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -bundle -undefined dynamic_lookup -L/Users/john/.pyenv/versions/3.7.9/lib -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -std=gnu++14 -L/usr/local/opt/readline/lib -L/Users/john/.pyenv/versions/3.7.9/lib -L/usr/local/opt/tcl-tk/lib build/temp.macosx-11.1-x86_64-3.7/pparser.o -o build/lib.macosx-11.1-x86_64-3.7/pparser.cpython-37m-darwin.so --verbose

and I see the following:

InstalledDir:     /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x86_64 -bundle -platform_version macos 11.0.0 11.3 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -undefined dynamic_lookup -undefined dynamic_lookup -o build/lib.macosx-11.1-x86_64-3.7/pparser.cpython-37m-darwin.so -L/Users/john/.pyenv/versions/3.7.9/lib -L/usr/local/opt/readline/lib -L/Users/john/.pyenv/versions/3.7.9/lib -L/usr/local/opt/tcl-tk/lib build/temp.macosx-11.1-x86_64-3.7/pparser.o -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.5/lib/darwin/libclang_rt.osx.a

I suspect that the problem is the ld switch -platform-version macOS 11.0.0 11.3 but where does that come from? It's not in the setup file or in the command line that I typed.

My set up is brew python 3.7.9 in a pyenv virtual environment.

Any help on understanding where python setup is pulling command line options from would be quite helpful.

ld macos python-3.x
1 Answers

After more digging, I found the answer here:

Customizing the compiler and linker used by setuptools

It seems that CPython creates a sysconfig file containing all the environment variables used to compile it. These are then used by python when a setup file is built.

You can review these variables in python:

import distutils.sysconfig
distutils.sysconfig.get_config_var()

The author goes on to describe how to override any variable desired.

user_108696440
Related