How does a threading.Thread yield the rest of its quantum in Python?

user_3691 0
Viewed 32413

I've got a thread that's polling a piece of hardware.

while not hardware_is_ready():
    pass
process_data_from_hardware()

But there are other threads (and processes!) that might have things to do. If so, I don't want to burn up cpu checking the hardware every other instruction. It's been a while since I've dealt with threading, and when I did it wasn't Python, but I believe most threading libraries have a yield function or something that allows a thread to tell the scheduler "Give the other threads a chance."

while not hardware_is_ready():
    threading.yield()          # This function doesn't exist.
process_data_from_hardware()

But I can't find any reference to something like this in the threading documentation. Python does have a yield statement, but I'm pretty sure that's something else entirely (to do with generators).

What's the correct thing to do here?

yield multithreading python
4 Answers

time.sleep(0) is sufficient to yield control -- no need to use a positive epsilon. Indeed, time.sleep(0) MEANS "yield to whatever other thread may be ready".

user_958100

Read up on the Global Interpreter Lock (GIL).

For example: http://jessenoller.com/2009/02/01/python-threads-and-the-global-interpreter-lock/

Also: http://www.pyzine.com/Issue001/Section_Articles/article_ThreadingGlobalInterpreter.html

Do this in your code if you must do Busy Waiting (e.g. polling a device).

time.sleep( 0.0001 )

This will yield to the thread scheduler.

Also, I collected some notes and references in http://homepage.mac.com/s_lott/iblog/architecture/C551260341/E20081031204203/index.html

SLott_106610

Why the yield in time.sleep(0) might not be enough:

I had a similar problem and ultimately time.sleep(0) did not work in all cases whereas time.sleep(0.0001) worked. In my case I use the standard vanilla cpython - which can be found at python.org. Looking at the implementation of the sleep function in the C code there is a tiny difference between a call of the sleep(0) and sleep(something_other_than_null).

Looking at pysleep in timemodule.c you will see that in the first case

if (ul_millis == 0 || !_PyOS_IsMainThread()) {
  Py_BEGIN_ALLOW_THREADS
  Sleep(ul_millis);
  Py_END_ALLOW_THREADS
  break;
}

is called. The "allow threads" releases the GIL, Sleep(0) in that case is equivalent to std::this_thread::yield(), after that the GIL is claimed again. In the code that deals with a non-zero value, apart from the actual sleep/wait there is an additional call to PyErr_CheckSignals - which as stated in its documentation "checks whether a signal has been sent to the processes and if so, invokes the corresponding signal handler". That would explain why in some cases when working with hardware a time.sleep(0) is not enough. So this might be a slip on cpython implementation side. And that is why time.sleep(0.0001) magically works.

user_20565450

If you're doing this on *nix, you might find the select library useful. Kamaela also has a few components you may find useful, but it may require a bit of a paradigm change.

user_21470
Related