How to efficiently share N selenium driver instances with N threads in python?

I want to share N selenium instances (instantiated before starting the threading process and stored in a list) between N threads in python.

I've been using ThreadPoolExecutor#map for multithreading, however this does not wait until a batch is finished - it simply continues as long as there's room for another thread (i.e depending on max_workers)

To demonstrate what I mean, let's take N = 3, i.e 3 threads

import time
from concurrent.futures import ThreadPoolExecutor

def foo(i: int):
    if i == 1:
        time.sleep(5)
    time.sleep(2)
    print(i)

with ThreadPoolExecutor(max_workers=3) as executor:
    executor.map(foo, [1, 2, 3, 4, 5, 6, 7, 8, 9])

One of the possible prints-

3
2
4
5
7
6
1
9
8

As is apparent, 1 was part of the first batch but since it took more time to finish than its colleagues in said batch, the threadpool simply continued - since it's allowed to still use 2 more threads (2 and 3 of first batch are finished).

This makes perfect sense, but is there anyway to run the multithreading in "batches". So that it stops until all of 1, 2 and 3 are finished before continuing?

My real problem, ofcourse, involves storing selenium instances in a list and turning it into a cycle, using itertools.cycle

from itertools import cycle
from selenium.webdriver.chrome.webdriver import WebDriver

# NOTE - N=3 for this example
drivers = cycle([Webdriver.Chrome() for _ in range(3)])

Now if I do-

# Assume foo is a function that uses a singular webdriver instance as well as a singular argument from argument_list
# This foo is not the same function as the previous dummy example

with ThreadPoolExecutor(max_workers=3) as executor:
    executor.map(foo, drivers, argument_list)

Given the first thread takes longer to finish, this is what will happen, I assume-

• First batch gets their own drivers, as there's one for each

• Thread 2 and Thread 3 finish, Thread 4 starts

  drivers is a cycle, the driver thread 4 will get is the same driver Thread 1 is still using

• Thread 1 gets screwed over

How can I avoid this? I don't want to initialize the webdrivers inside the function foo since this is supposed to be quite critical on performance.

Note: I should also mention, I am concerned about the return value of the function I'll call. Although not demonstrated in the examples, I'll be iterating through the result of executor.map and storing the results in a list. I know map follows order for results, but in case I've to use some other function to achieve what I want, I should mention - the order of the batches is important but not necessarily the unit results themselves.

What do I mean by this? If the result is returned like-

1st_result, 3rd_result, 2nd_result, 5th_result, 4th result, 6th result

for N = 3 (3 threads) and len(argument_list) = 6

that's fine. But if any of the results from the 2nd batch (4, 5, 6) is returned before all the results from the first batch have, that's bad.

e.g I don't want-

1st_result, 3rd_result, 5th result, 2nd_result, 4th result, 6th result

.map already does this fine, except I don't see any way to make it wait before starting the next batch and potentially screwing over a thread