Why am I able to run more concurrent threads than I have cpus?

user_4391249 0
Viewed 63

My CPU has 4 cores so as I understand threads in Python, this means I can only run 4 threads concurrently. I wanted to test this out so I wrote:

import threading
from time import sleep

def print_after_sleep(msg):
    sleep(5)
    print(msg)

one = threading.Thread(target=print_after_sleep, args=[1])
two = threading.Thread(target=print_after_sleep, args=[2])
three = threading.Thread(target=print_after_sleep, args=[3])
four = threading.Thread(target=print_after_sleep, args=[4])
five = threading.Thread(target=print_after_sleep, args=[5])
six = threading.Thread(target=print_after_sleep, args=[6])
one.start()
sleep(0.2)
two.start()
sleep(0.2)
three.start()
sleep(0.2)
four.start()
sleep(0.2)
five.start()
sleep(0.2)
six.start()

I would expect threads 1-4 to print their messages in rapid succession, but then to wait nearly 5 seconds for threads 5-6 to print their messages. What I actually got, was all threads printing their messages in rapid succession.

multithreading python
2 Answers

Let's talk about threads first.

There are almost always more threads in a modern day application than there are cores in the machine's CPU. Each thread gets assigned a particular core. The core then executes each thread in a time-shared fashion. The thread to be executed is decided by the Operating System's scheduler.

Your misconception here is the amount of time the scheduler takes to switch between threads. The switch is done in the order of microseconds (1e-6 seconds), AT MOST. Taking 5-6 seconds to switch threads will render most software today useless.

Coming to specifics about python, the language is restricted by a Global Interpreter Lock, or GIL. In short, this may cause all your threads to run on a single core.

EDIT

Threads have different states, that help the OS scheduler optimize their operations.

Thread states

I'm not going in the details of the above diagram. All you need the know are the following:

  1. Multiple threads can be in the 'ready' state in a single core. This essentially means that they can be executed when the scheduler allows them.

  2. Only one thread per core can be 'running'. All other threads will be 'ready' or 'blocked' (aka 'waiting').

  3. When you call sleep, the thread basically moves into a different state. The moment there is no 'running' thread, the scheduler will immediately assign a 'ready' thread to run. In your case, this is a thread that has finished waiting for 5 seconds.

In your program, the naive interpretation will be that all threads are 'blocked' for 5 seconds. The core assigned is probably not even running your program at the time. Then, all threads simultaneously move to 'ready' state, where the scheduler executes them one at a time.

user_55339280

This analogy might help:

Imagine a restaurant where there are only two waiters but several customers (up to the maximum seating capacity of 50).

At any given time, there may be up to 50 customers dining concurrently.

How is that possible despite there being only two waiters?

The answer is, of course:

  1. Not every customer needs the attention / services of a waiter at every instant of time. There are times when some customers are eating or conversing with each other or on their phones, and they most certainly don't want a waiter at those times.
  2. If a customer wants a waiter to take their order, it is possible that both the waiters are busy -- in which case the customer would wait for the waiter.

The same thing happens in the case of concurrent threads running on a limited number of cores or processors. The threads running concurrently corresponds to the customers dining concurrently in a restaurant. The limited number of cores/processors corresponds to the limited number of waiters. The threads not needing to use a core while waiting for an I/O operation, corresponds to a customer not needing a waiter as they are busy conversing or eating. A customer occasionally having to wait for a waiter corresponds to those rare situations when a thread starves for processor time (that is, a thread wants to run, but the limited cores are busy running other threads). A customer may sometimes instruct the waiter to leave them alone for, say 5 minutes, for making up their mind on their order. That could correspond to a thread invoking the sleep() function.

Hope this much answers the question in your title.

As for the question in the body of your post, I suppose your expectation was based on the wrong understanding that sleep() puts the the core to sleep. It is actually the invoking thread that goes to sleep -- leaving the core with one less thread to serve during that time.

user_85619570
Related