Splitting large list into chunks using modulos

user_10704286 0
Viewed 26

I'm wondering why the simple code below doesn't work:

my_list = range(0, 6)

num_proc = 3

chunks = [[]] * num_proc

i = 0
for item in my_list:
    idx = i % num_proc
    i += 1
    chunks[idx].append(item)

print(f"Chunked items = {chunks}")

It outputs:

Chunked items = [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]

The output I expect is

Chunked items = [[0, 3], [1, 4], [2, 5]]

Seems that the problem lies with statement

 chunks[idx].append(item)

Yet, I'm at a loss on how to fix that

python
2 Answers

This is happening because [[]] * num_proc creates a reference of the same list num_proc times so any change to one them updates all of them.

So instead of:

chunks = [[]] * num_proc

Do:

chunks = [[] for _ in range(num_proc)]

user_180085150

The reason is because when you create your chunkslist you create 3 sub-list with the same reference. Therefore when you update it in your code the three sub-lists are updated together.

This line should fix it, it implements chunks in a different way :

chunks = [[] for i in range(num_proc)]

user_149773680
Related