Embedded Python in C program seg faults when ran from script in python

Evgeny Foox 0
Viewed 48

I wrote a short program trying to embed python into it (according to the C Python Api docs : https://docs.python.org/3/extending/embedding.html):

#include <python3.8/Python.h>
#include <stdio.h>

extern "C"
{
const char *buf1 = "def func():\n"
"    print(\"hello\")\n";

int Func(void)
{
    if (0 == Py_IsInitialized())
    {
        Py_Initialize();
    }

    printf("%s", buf1);

    PyObject* pPythonScript = Py_CompileString(buf1 , "xx" , Py_single_input);

    Py_DecRef(pPythonScript);

    Py_FinalizeEx();

    return (0);
}
}

I compiled it into a shared object:

gcc -fpic test.cpp $(python3.8-config --embed --ldflags) -c -I/usr/include/python3.8 -g
gcc -shared -o test.so test.o

Then I tried running this function from python:

from ctypes import *

path = "<abs path to shared obj>/test.so"

shared_obj = CDLL(path)

shared_obj.Func()

Ran the python program

$ python <prog name>.py

and it resulted with a segfault:

def func():
    print(hello)
Segmentation fault (core dumped)

Now, I've tried running the C program only (Changing the Func name into main), and all ran alright.

I ran the python program with gdb

gdb --args python test.py

and got the following:

Starting program: /usr/bin/python test.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
def func():
    print(hello)

Program received signal SIGSEGV, Segmentation fault.
0x000000000058b3b6 in PyUnicode_DecodeFSDefaultAndSize ()

The documentation on PyUnicode_DecodeFSDefaultAndSize yielded nothing but I know that I'm missing something, but according to the docs, it should work.

embedding python-3.x c c++ python
1 Answers

I've managed to solve the issue by adding a GIL lock:

#include <python3.8/Python.h>
#include <stdio.h>

extern "C"
{
const char *buf1 = "def func():\n"
"    print(\"hello\")\n";

int Func(void)
{
    PyGILState_STATE gilstate;

    if (0 == Py_IsInitialized())
    {
        Py_Initialize();
    }

    gilstate = PyGILState_Ensure();

    printf("%s", buf1);

    PyObject* pPythonScript = Py_CompileString(buf1 , "xx" , Py_single_input);

    Py_DecRef(pPythonScript);

    PyGILState_Release(gilstate);

    return (0);
}
}
Evgeny Foox0
Related