Python is not freeing memory

Question

I've been working with XML resources, and it seems that Python is issuing a weird behavior. I've tested both lxml library and xml.etree.ElementTree, both holding memory after it should be collected by gc. I typed gc.collect() as a test, but nothing else happen: memory still being held by process.

Imports:

import time
from lxml import etree
import gc

This is the code:

def process_alternative():
    """
    This alternative process will use lxml
    """
    filename = u"/tmp/randomness.xml"
    fd = open(filename, 'r')
    tree = etree.parse(fd)
    root = tree.getroot()

    accum = {}

    for _item in root.iter("*"):
        for _field in _item.iter("*"):
            if _field.tag in accum.keys():
                accum[_field.tag] += 1
            else:
                accum[_field.tag] = 1

    for key in accum.keys():
        print "%s -> %i" % (key, accum[key])

    fd.close()
    gc.collect()

And this is my main

if __name__ == "__main__":
    while True:
        print "Wake up!"
        process_alternative()
        print "Sleeping..."
        time.sleep(30)

As you see, this main calls "process_alternative", and then sleep. XML file provided loads memory with nearly 800MB; so, before time.sleep, memory should be freed by process, returning to basic VM memory needed (around 32MB?). Instead, process continue holding around 800MB.

Any tip about why memory has not been freed after every iteration?

Using ubuntu 13.04, Python 2.7.4

This function deallocates memory in every iteration

def check_memory():
    ac1 = [a1**5 for a1 in xrange(10000000)]
    time.sleep(5)
    ac2 = [a1**5 for a1 in xrange(10000000)]
    time.sleep(5)
    ac3 = [a1**5 for a1 in xrange(10000000)]

Python is not freeing memory

1 Answers