Is Python list slicing thread safe?

Viewed 148

Coming from Are lists thread-safe?, I need to know if specifically list slicing is thread safe. It's not clear to me from the linked article What kinds of global value mutation are thread-safe?.

Based on the answer to is list.pop thread safe in python, it seems I really need to know if list slicing an atomic operation.

I have a list some_list that another thread is continuously appending to. In the main thread, I sometimes peek (without poping) at:

  • The last element of the list like so: some_list[-1] (indexing)
  • Last couple of elements of the list like so: some_list[-3:] (slicing)

So, the question is, in CPython (I use Python 3.10), is list slicing thread-safe?

1 Answers

tl;dr both indexing and slicing are thread safe, but slicing consist of building slice and applying slice. Between them thread can modify our list.

In stackoverflow page you posted you can find a list of atomic operations for iterables and both, indexing and slicing are there.

Lets look at bytecode for slicing:

import dis

l = [1, 2, 3]


def f():
    l[1:3]


dis.dis(f)
  7           0 LOAD_GLOBAL              0 (l)
              2 LOAD_CONST               1 (1)
              4 LOAD_CONST               2 (3)
              6 BUILD_SLICE              2
              8 BINARY_SUBSCR
              (...)

BINARY_SUBSCR is a moment when you access the slice and it's indeed single operation. Check that:

import time
import threading

l = [0] * 1000000000


def append():
    start = time.perf_counter()
    time.sleep(0.1)
    l.append(1)  # Doesn't execute until main thread access slice
    print("append", time.perf_counter() - start)  # 3.4184917740058154


t = threading.Thread(target=append)
t.start()
start = time.perf_counter()
x = l[:]  # Reading via slicing
print("main", time.perf_counter() - start)  # 3.418436762993224
print("last_elem", x[-1])  # 0
t.join()

So fear not of changes in list while accessing a slice (and same with accessing an index) as it's simple opcode and is protected by GIL.

What can happen then?

# l = [1, 2, 3]
x = l[0:3]

You cannot assume x will contain all list.

  1. We first (BUILD_SLICE) with start 0, end 3.
  2. Then another thread can modify list e.g. by appending element.
  3. Only then we access slice with thread safe way.

Can it make our program crash? I doubt, as using out of bounds slice is safe.

Related