Segmentation fault on requests.request

Question

I am getting a segmentation fault after using python requests library to connect to Twitter's v2 API endpoint.

The request is created on a separate thread in a generator object

class TwitterStream:
    def __init__():
        self.response = self.build_response()    

    def build_response():
        response = requests.request("GET", url, headers, stream=True)
        if response.status_code != 200:
            raise Exception("Exception")
        for record in response.iter_lines():
            yield record

    def __next__():
        return next(self.response)

Then on the execution thread, I use a ThreadPoolExecutor to get each response line:

record_future = threadpoolexecutor.submit(next, TwitterStreamObjectInstance)
if record_future.done():
   # do stuff  with record_future.result()

I keep getting a segmentation fault at the response = requests.request("GET", url, headers, stream=True) line, I'm pretty sure, based on a lot of print() debugging.

I tried the gdb debugging method, and got back this stacktrace -- I am assuming libssl.so.1 is causing this problem, but I don't really know how to investigate this further.

#0  0x000000000001d85e in ?? ()
#1  0x00007ffff406ffa0 in OPENSSL_init_ssl () from /mnt/d/Projects/EBKA/edna_env/lib/python3.7/site-packages/mysql/vendor/libssl.so.1.1
#2  0x00007ffff4074be3 in SSL_CTX_new () from /mnt/d/Projects/EBKA/edna_env/lib/python3.7/site-packages/mysql/vendor/libssl.so.1.1
#3  0x00007ffff43c855d in ?? () from /usr/lib/python3.7/lib-dynload/_ssl.cpython-37m-x86_64-linux-gnu.so
#4  0x00005555556c9ff2 in ?? ()
#5  0x00005555556857d0 in _PyMethodDef_RawFastCallKeywords ()
#6  0x0000555555685560 in _PyCFunction_FastCallKeywords ()
#7  0x00005555556fa4e4 in _PyEval_EvalFrameDefault ()
#8  0x00005555556f4e2f in _PyEval_EvalCodeWithName ()
#9  0x0000555555687591 in _PyObject_Call_Prepend ()
#10 0x00005555556c9e3c in ?? ()
#11 0x00005555556c5ce5 in ?? ()
#12 0x0000555555685f72 in _PyObject_FastCallKeywords ()
#13 0x00005555556fa00d in _PyEval_EvalFrameDefault ()
#14 0x00005555556f4e2f in _PyEval_EvalCodeWithName ()
#15 0x0000555555686efa in _PyFunction_FastCallKeywords ()
#16 0x00005555556f6b0a in _PyEval_EvalFrameDefault ()
#17 0x0000555555686e1a in _PyFunction_FastCallKeywords ()
.
.
.
#58 0x00005555555c5d24 in _PyFunction_FastCallDict ()
#59 0x00005555556f73d0 in _PyEval_EvalFrameDefault ()
#60 0x0000555555686e1a in _PyFunction_FastCallKeywords ()
#61 0x00005555556f5fde in _PyEval_EvalFrameDefault ()
#62 0x0000555555686e1a in _PyFunction_FastCallKeywords ()
#63 0x00005555556f5fde in _PyEval_EvalFrameDefault ()
#64 0x0000555555687359 in _PyObject_Call_Prepend ()
#65 0x00005555556879b8 in PyObject_Call ()
#66 0x0000555555800193 in ?? ()
#67 0x00005555557c30d7 in ?? ()
#68 0x00007ffff7bbb6db in start_thread (arg=0x7fffd0f92700) at pthread_create.c:463
#69 0x00007ffff6cf071f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

A side note -- I have been having a problem with closing threads, where I get the following

libgcc_s.so.1 must be installed for pthread_cancel to work

A suggestion I got was to do the following in the beginning of execution:

import ctypes
libgcc_s = ctypes.CDLL('libgcc_s.so.1')

but this didn't work. The reason I mention this is because after I tried this, I kept getting a "cannot find libssl" error, and I am not sure if it would be related.

Employed_Russian_50617 · Answer

In my experience crashes in OPENSSL_init_ssl most often result from linking in more than one version (or more than one instance) of libssl.so.x.y into a single process.

Use GDB info shared libssl at crash point.

If you see more than one version, that's very likely the root cause.

If you don't, it's still possible that some of your binaries link in libssl.a. For each binary (the main python executable and every shared library in info shared output), run nm -A python lib1.so lib2.so ... | grep OPENSSL.

If the only matching library is /mnt/d/Projects/EBKA/edna_env/lib/python3.7/site-packages/mysql/vendor/libssl.so.1.1, then there must be some other root cause.

Segmentation fault on requests.request

1 Answers