Check what a running process is doing: print stack trace of an uninstrumented Python program

user_189247 0
Viewed 11361

Is there a way on Linux to check what a running Python daemon process is doing? That is, without instrumenting the code and without terminating it? Preferably I'd like to get the name of the module and the line number in it that is currently running.

Conventional debugging tools such as strace, pstack and gdb are not very useful for Python code. Most stack frames just contain functions from the interpreter code like PyEval_EvalFrameEx and and PyEval_EvalCodeEx, it doesn't give you any hint on were in the .py-file the execution is.

daemon process stack-trace python debugging
8 Answers

py-spy (https://github.com/benfred/py-spy) has a few useful tools for inspecting running Python processes. In particular, py-spy dump will print a stack trace (including function, file, and line) for every thread.

user_494850

You can use madbg (by me). It is a python debugger that allows you to attach to a running python program and debug it in your current terminal. It is similar to pyrasite and pyringe, but newer, doesn't require gdb, and uses IPython for the debugger (which means colors and autocomplete).

To see the stack trace of a running program, you could run:

madbg attach <pid>

And in the debugger shell, enter: bt

user_29078190

It's possible to debug Python with gdb. See Chapter 22: gdb Support in the Python Developer’s Guide.

For example, on Debian with Python 3.7:

# apt-get update -y && apt-get install gdb python3.7-dbg
# gdb

(gdb) source /usr/share/gdb/auto-load/usr/bin/python3.7-gdb.py
(gdb) attach <PID>
(gdb) py-bt
user_5942110

You can also use satella to do this. A nice side effect will be that every local variable in every stack frame will be printed out. The code would be:

from satella.instrumentation import Traceback
import sys

for frame_no, frame in sys._current_frames().items():
    sys.stderr.write("For stack frame %s" % (frame_no,))
    tb = Traceback(frame)
    tb.pretty_print()
sys.stderr.write("End of stack frame dump\n")
user_12081330
Related