Orphan stacktraces in sys._current_frames()

user_76701 0
Viewed 163

Here's an esoteric pure-Python question.

I'm doing some statistical profiling using sys._current_frames(). i.e. I've got a background thread that runs sys._current_frames() once every second, dumps the results in a text file, and then later I've got some Python code that sorts the tracebacks from most common to least.

One curious phenomenon I've seen is tracebacks like these:

  File "/opt/foo/bar.py", line 1437, in __iter__
    yield key

This yield is a generator that I wrote. The curious thing is that there's just one frame on this traceback. How could this be? The other traceback have lots of frames, either from the top level of the process or the top level of the frame. What is the meaning of this single-frame stacktrace?

One theory that I had is that this is a generator's frozen state, after it's yielded a value and it's waiting to have next called on it again. But I think I disproved this theory with a separate experiment: I made a generator, ensured it was paused, called sys._current_frames() and I didn't see that kind of stacktrace.

introspection generator stack-trace python
1 Answers

As the sys._current_frames() documentation warns,

This is most useful for debugging deadlock: this function does not require the deadlocked threads’ cooperation, and such threads’ call stacks are frozen for as long as they remain deadlocked. The frame returned for a non-deadlocked thread may bear no relationship to that thread’s current activity by the time calling code examines the frame.

sys._current_frames() is naturally prone to race conditions in any situation where you cannot guarantee the threads of interest are paused.

As you suspected, you're seeing a stack trace for a suspended generator. When a generator suspends, its stack frame has no parent frames. Its f_back is set to null.

sys._current_frames() retrieves stack frames for currently running threads, but by the time you look at those frames, they may not be running any more. If a generator suspends between the time you call sys._current_frames() and the time you inspect the frame, this is what it will look like. You might also see it on top of a call stack that looks completely different from when you actually called sys._current_frames(), if it gets resumed somewhere else.

Your test didn't show the generator frame because you suspended the generator before calling sys._current_frames() instead of afterward. The generator's stack frame was not the active frame of any thread at that point.

user_23571120
Related