Processing images on the server with pytorch and facing memory issues. In particular:
Scenario 1:
#before execution memory used is 1518
temp = PIL.Image.open(im) #no leak
del im
del temp
gc.collect()
#after execution x5 times memory used is 1518
Scenario 2:
#before execution memory used is 1518
temp = PIL.Image.open(im) #no leak
del im
temp2 = np.array(temp) #leak? (asarray has same behaviour)
del temp
temp2 = None
del temp2
gc.collect()
#after execution x5 times memory used is 2385
Details:
numpy==1.19.0
python==3.6.9
flask
apache2
libapache2-mod-wsgi-py3
virtualenv
watching memory using 'watch free -m'
I do not think this is flask or wsgi -related
However, if I do a simple experiment in python3 environment:
#before execution memory used is 2418
a = [random.random()]*100000000
b = np.array(a)
del a
del b
#after execution memory used is 2418