Disable output buffering

Viewed 350419

Is output buffering enabled by default in Python's interpreter for sys.stdout?

If the answer is positive, what are all the ways to disable it?

Suggestions so far:

  1. Use the -u command line switch
  2. Wrap sys.stdout in an object that flushes after every write
  3. Set PYTHONUNBUFFERED env var
  4. sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

Is there any other way to set some global flag in sys/sys.stdout programmatically during execution?

16 Answers

From Magnus Lycka answer on a mailing list:

You can skip buffering for a whole python process using python -u (or #!/usr/bin/env python -u etc.) or by setting the environment variable PYTHONUNBUFFERED.

You could also replace sys.stdout with some other stream like wrapper which does a flush after every call.

class Unbuffered(object):
   def __init__(self, stream):
       self.stream = stream
   def write(self, data):
       self.stream.write(data)
       self.stream.flush()
   def writelines(self, datas):
       self.stream.writelines(datas)
       self.stream.flush()
   def __getattr__(self, attr):
       return getattr(self.stream, attr)

import sys
sys.stdout = Unbuffered(sys.stdout)
print 'Hello'
# reopen stdout file descriptor with write mode
# and 0 as the buffer size (unbuffered)
import io, os, sys
try:
    # Python 3, open as binary, then wrap in a TextIOWrapper with write-through.
    sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True)
    # If flushing on newlines is sufficient, as of 3.7 you can instead just call:
    # sys.stdout.reconfigure(line_buffering=True)
except TypeError:
    # Python 2
    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

Credits: "Sebastian", somewhere on the Python mailing list.

Yes, it is.

You can disable it on the commandline with the "-u" switch.

Alternatively, you could call .flush() on sys.stdout on every write (or wrap it with an object that does this automatically)

This relates to Cristóvão D. Sousa's answer, but I couldn't comment yet.

A straight-forward way of using the flush keyword argument of Python 3 in order to always have unbuffered output is:

import functools
print = functools.partial(print, flush=True)

afterwards, print will always flush the output directly (except flush=False is given).

Note, (a) that this answers the question only partially as it doesn't redirect all the output. But I guess print is the most common way for creating output to stdout/stderr in python, so these 2 lines cover probably most of the use cases.

Note (b) that it only works in the module/script where you defined it. This can be good when writing a module as it doesn't mess with the sys.stdout.

Python 2 doesn't provide the flush argument, but you could emulate a Python 3-type print function as described here https://stackoverflow.com/a/27991478/3734258 .

Yes, it is enabled by default. You can disable it by using the -u option on the command line when calling python.

In Python 3, you can monkey-patch the print function, to always send flush=True:

_orig_print = print

def print(*args, **kwargs):
    _orig_print(*args, flush=True, **kwargs)

As pointed out in a comment, you can simplify this by binding the flush parameter to a value, via functools.partial:

print = functools.partial(print, flush=True)

You can create an unbuffered file and assign this file to sys.stdout.

import sys 
myFile= open( "a.log", "w", 0 ) 
sys.stdout= myFile

You can't magically change the system-supplied stdout; since it's supplied to your python program by the OS.

One way to get unbuffered output would be to use sys.stderr instead of sys.stdout or to simply call sys.stdout.flush() to explicitly force a write to occur.

You could easily redirect everything printed by doing:

import sys; sys.stdout = sys.stderr
print "Hello World!"

Or to redirect just for a particular print statement:

print >>sys.stderr, "Hello World!"

To reset stdout you can just do:

sys.stdout = sys.__stdout__
Related