PyGILState_Ensure() Causing Deadlock

user_3350050 0
Viewed 4356

I'm writing a Python extension in C++, wrapping a third-party library I do not control. That library creates a thread Python knows nothing about, and from that thread, calls a C++ callback I provide to the library. I want that callback to call a Python function, but I get a deadlock using the approach I read from the docs. Here's my interpretation of those.

void Wrapper::myCallback()
{
   PyGILState_STATE gstate=PyGILState_Ensure();
   PyObject *result=PyObject_CallMethod(_pyObj,"callback",nullptr);
   if (result) Py_DECREF(result);
   PyGILState_Release(gstate);
}

My code does nothing else related to threads, though I've tried a number of other things that have. Based on this, for example, I tried calling PyEval_InitThreads(), but it's not obvious where that call should be made for an extension. I put it in the PyMODINIT_FUNC. These attempts have all lead to deadlock, crashes, or mysterious fatal errors from Python, e.g., PyEval_ReleaseThread: wrong thread state.

This is on Linux with Python 3.6.1. Any ideas how I can get this "simple" callback to work?

Likely Culprit

I didn't realize that in another thread, the library was in a busy/wait loop waiting on the callback's thread. In gdb, info threads made this apparent. The only solution I can see is to skip those particular calls to the callback; I don't see a way to make them safe, given the busy/wait loop. In this case, that's acceptable, and doing so eliminates the deadlock.

Also, it appears that I do need to also call PyEval_InitThreads() before any of this. In a C++ extension, it's not clear where that should go though. One of the replies suggested doing it indirectly in Python by creating and deleting a throwaway threading.Thread. That didn't seem to fix it, triggering instead a Fatal Python error: take_gil: NULL tstate, which I think means there's still no GIL. My guess, based on this and the issue it refers to, is that PyEval_InitThreads() causes the current thread to become the main thread for the GIL. If that call is made in the short-lived throwaway thread, maybe that's a problem. Yeah, I'm only guessing and would appreciate an explanation from someone who doesn't have to.

deadlock multithreading python c++
3 Answers

This answer is only for Python >= 3.0.0. I don't know if it would work for earlier Pythons or not.

Wrap your C++ module in a Python module that looks something like this:

import threading
t = threading.Thread(target=lambda: None, daemon=True)
t.run()
del t
from your_cpp_module import *

From my reading of the documentation, that should force threading to be initialized before your module is imported. Then the callback function you have written up there should work.

I'm less confident of this working, but your module init function could instead do this:

if (!PyEval_ThreadsInitialized())
{
    PyEval_InitThreads();
}

that should work because your module init function should be being executed by the only Python thread in existence if PyEval_ThreadsInitialized() isn't true, and holding the GIL is the right thing to do then.

These are guesses on my part. I've never done anything like this as is evidenced by my clueless comments on your question. But from my reading of the documentation, both of these approaches should work.

user_1679580

I have wrapped C++ observers in Python. If you are using boost then you can call PyEval_InitThreads() in BOOST_PYTHON_MODULE:

BOOST_PYTHON_MODULE(eapipy)
{
     boost::shared_ptr<Python::InitialisePythonGIL> gil(new Python::InitialisePythonGIL());
....
}

Then I use a class to control calling back into Python from C++.

struct PyLockGIL
{

    PyLockGIL()
        : gstate(PyGILState_Ensure())
    { 
    }

    ~PyLockGIL()
    {
        PyGILState_Release(gstate);
    }

    PyLockGIL(const PyLockGIL&) = delete;
    PyLockGIL& operator=(const PyLockGIL&) = delete;

    PyGILState_STATE gstate;
};

If you are calling into C++ for any length of time you can also relinquish the GIL:

struct PyRelinquishGIL
{
    PyRelinquishGIL()
        : _thread_state(PyEval_SaveThread())
    {
    }
    ~PyRelinquishGIL()
    {
        PyEval_RestoreThread(_thread_state);
    }

    PyRelinquishGIL(const PyLockGIL&) = delete;
    PyRelinquishGIL& operator=(const PyLockGIL&) = delete;

    PyThreadState* _thread_state;
};

Our code is multi-threaded and this approach works well.

user_52672510
Related