Threadsafe and fault-tolerant file writes

Viewed 5425

I have a long-running process which writes a lot of stuff in a file. The result should be everything or nothing, so I'm writing to a temporary file and rename it to the real name at the end. Currently, my code is like this:

filename = 'whatever'
tmpname = 'whatever' + str(time.time())

with open(tmpname, 'wb') as fp:
    fp.write(stuff)
    fp.write(more stuff)

if os.path.exists(filename):
    os.unlink(filename)
os.rename(tmpname, filename)

I'm not happy with that for several reasons:

  • it doesn't clean up properly if an exception occurs
  • it ignores concurrency issues
  • it isn't reusable (I need this in different places in my program)

Any suggestions how to improve my code? Is there a library that can help me out?

4 Answers
Related