I am building a module in C++ to be used in Python. My flow is three steps: I compile the individual C++ sources into objects, create a library, and then run a setup.py script to compile the .pyx->.cpp->.so, while referring to the library I just created.
I know I can just do everything in one step with the Cython setup.py, and that is what I used to do. The reason for splitting it into multiple steps is I'd like the C++ code to evolve on its own, in which case I would just use the compiled library in Cython/python.
So this flow works fine, when there are no bugs. The issue is I am trying to find the source of a segfault, so I'd like to get the debugging symbols so that I can run with gdb (which I installed on OSX 10.14, it was a pain but it worked).
I have a makefile, which does the following.
Step 1: Compile individual C++ source files
All the files are compiled with the bare minimum flags, but -g is there:
gcc -mmacosx-version-min=10.7 -stdlib=libc++ -std=c++14 -c -g -O0 -I ./csrc -o /Users/colinww/system-model/build/data_buffer.o csrc/data_buffer.cpp
I think even here there is a problem: when I do nm -pa data_buffer.o, I see no debug symbols. Furthermore, I get:
(base) cmac-2:system-model colinww$ dsymutil build/data_buffer.o
warning: no debug symbols in executable (-arch x86_64)
Step 2: Compile cython sources
The makefile has the line
cd $(CSRC_DIR) && CC=$(CC) CXX=$(CXX) python3 setup_csrc.py build_ext --build-lib $(BUILD)
The relevant parts of setup.py are
....
....
....
compile_args = ['-stdlib=libc++', '-std=c++14', '-O0', '-g']
link_args = ['-stdlib=libc++', '-g']
....
....
....
Extension("circbuf",
["circbuf.pyx"],
language="c++",
libraries=["cpysim"],
include_dirs = ['../build'],
library_dirs=['../build'],
extra_compile_args=compile_args,
extra_link_args=link_args),
....
....
....
ext = cythonize(extensions,
gdb_debug=True,
compiler_directives={'language_level': '3'})
setup(ext_modules=ext,
cmdclass={'build_ext': build_ext},
include_dirs=[np.get_include()])
When this is run, it generates a bunch of compilation/linking commands like
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/colinww/anaconda3/include -arch x86_64 -I/Users/colinww/anaconda3/include -arch x86_64 -I. -I../build -I/Users/colinww/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/Users/colinww/anaconda3/include/python3.7m -c circbuf.cpp -o build/temp.macosx-10.7-x86_64-3.7/circbuf.o -stdlib=libc++ -std=c++14 -O0 -g
and
g++ -bundle -undefined dynamic_lookup -L/Users/colinww/anaconda3/lib -arch x86_64 -L/Users/colinww/anaconda3/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.7/circbuf.o -L../build -lcpysim -o /Users/colinww/system-model/build/circbuf.cpython-37m-darwin.so -stdlib=libc++ -g
In both commands, the -g flag is present.
Step 3: Run debugger
Finally, I run my program with gdb
(base) cmac-2:sim colinww$ gdb python3
(gdb) run system_sim.py
It dumps out a ton of stuff related to system files (seems unrelated) and finally runs my program, and when it segfaults:
Thread 2 received signal SIGSEGV, Segmentation fault.
0x0000000a4585469e in cpysim::DataBuffer<double>::Write(long, long, double) () from /Users/colinww/system-model/build/circbuf.cpython-37m-darwin.so
(gdb) info local
No symbol table info available.
(gdb) where
#0 0x0000000a4585469e in cpysim::DataBuffer<double>::Write(long, long, double) () from /Users/colinww/system-model/build/circbuf.cpython-37m-darwin.so
#1 0x0000000a458d6276 in cpysim::ChannelFilter::Filter(long, long, long) () from /Users/colinww/system-model/build/chfilt.cpython-37m-darwin.so
#2 0x0000000a458b0d29 in __pyx_pf_6chfilt_6ChFilt_4filter(__pyx_obj_6chfilt_ChFilt*, long, long, long) () from /Users/colinww/system-model/build/chfilt.cpython-37m-darwin.so
#3 0x0000000a458b0144 in __pyx_pw_6chfilt_6ChFilt_5filter(_object*, _object*, _object*) () from /Users/colinww/system-model/build/chfilt.cpython-37m-darwin.so
#4 0x000000010002f1b8 in _PyMethodDef_RawFastCallKeywords ()
#5 0x000000010003be64 in _PyMethodDescr_FastCallKeywords ()
As I mentioned above, I think the problem starts in the initial compilation step. This has nothing to do with cython, I'm just calling gcc from the command line, passing the -g flag.
(base) cmac-2:system-model colinww$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Any help is appreciated, thank you!
UPDATE
I removed the gcc tag and changed it to clang. So I guess now I'm confused, if Apple will alias gcc to clang, doesn't that imply that in "that mode" it should behave like gcc (and, implied, someone made sure it was so).
UPDATE 2
So, I never could get the debug symbols to appear in the debugger, and had to resort to lots of interesting if-printf statements, but the problem was due to an index variable becoming undefined. So thanks for all the suggestions, but the problem is more or less resolved (until next time). Thanks!