charm.pool.map & tqdm: obtain a progressbar

user_8425270 0
Viewed 146

I'm using the wanderfull library charm4py to parralelise tasks on a cluster of several machines. I'm just using the function charm.pool.map, which is documented there.

Using the classical list(tqdm.tqdm(char.pool.map(...),total=...)) did not work: the progressbar was printed but only after the last iteration.

How should i write this ? is it even possible ?

Edit: cross-posted on a charm4py issue there : https://github.com/UIUC-PPL/charm4py/issues/178

charm++ tqdm python
1 Answers

Following @lrnv suggestion about multi_future, the following code should correctly display the progress of a pool of workers.

from charm4py import charm
from tqdm import tqdm

futures = charm.pool.map_async(func, iterable, multi_future=True)
pBar = tqdm(total=len(futures))
for future in charm.iwait(futures):
    # Do something with future.get()
    pBar.update()
user_106405340
Related