Is it better to pre-allocate array in python or use arr.append()?

user_14170759 0
Viewed 1034

In terms of readability and performance, should I pre-allocate memory for an array using [None]*n? Is allocating an empty one [] and using .append() over and over considered wasteful?

python performance
1 Answers

In this simple timing test, the use of [None] * n does indeed appear to be slightly quicker, but arguably not by enough to justify adopting this approach over the more usual idioms.

import time

def func1(size):
    a = [None] * size
    for i in range(size):
        a[i] = i

def func2(size):
    a = []
    for i in range(size):
        a.append(i)

def func3(size):
    a = [i for i in range(size)]

    
size = 1000000
repeat = 100
    
t0 = time.time()

for _ in range(repeat):
    func1(size)
t1 = time.time()

for _ in range(repeat):
    func2(size)
t2 = time.time()

for _ in range(repeat):
    func2(size)
t3 = time.time()

print(t1 - t0, t2 - t1, t3 - t2)

Results:

  • [None * size] and then index: 4.82 seconds
  • append in a loop: 6.37 seconds
  • list comprehension: 6.34 seconds

Repeating the tests with size=1000 and repeat=100000 give similar results:

  • [None * size] and then index: 3.16 seconds
  • append in a loop: 4.88 seconds
  • list comprehension: 4.84 seconds

And again with size=10 and repeat = 10000000:

  • [None * size] and then index: 6.09 seconds
  • append in a loop: 7.65 seconds
  • list comprehension: 7.66 seconds
user_135960370
Related