import side effects on logging: how to reset the logging module?

user_238671 0
Viewed 20583

Consider this code:

import logging
print "print"
logging.error("log")

I get:

print
ERROR:root:log

now if I include a thid-party module at the beginning of the previous code and rerun it I get only:

print

there are some previous question about this, but here I cannot touch the module I'm importing.

The code of the third-party module is here: http://atlas-sw.cern.ch/cgi-bin/viewcvs-atlas.cgi/offline/DataManagement/DQ2/dq2.clientapi/lib/dq2/clientapi/DQ2.py?view=markup, but my question is more general: independently of the module I'm importing I want a clean logging working in the expected way

Some (non-working) proposed solutions:

from dq2.clientapi.DQ2 import DQ2
import logging
del logging.root.handlers[:]

from dq2.clientapi.DQ2 import DQ2
import logging
logging.disable(logging.NOTSET)

logs = logging.getLogger('root')
logs.error("Some error")

the next one works, but produced some additional errors:

from dq2.clientapi.DQ2 import DQ2
import logging
reload(logging)

I get:

print
ERROR:root:log
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-    opt/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 1509, in shutdown
    h.close()
  File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 705, in close
    del _handlers[self]
KeyError: <logging.StreamHandler instance at 0x2aea031f7248>
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 1509, in shutdown
    h.close()
  File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 705, in close
    del _handlers[self]
KeyError: <logging.StreamHandler instance at 0x2aea031f7248>

from dq2.clientapi.DQ2 import DQ2
import logging
logger = logging.getLogger(__name__)
ch = logging.StreamHandler()  
logger.addHandler(ch)
logger.error("log")
import logging python
3 Answers

A more complete solution that doesn't invalidate any loggers. Should work, unless some module does something strange like holding a reference to a filterer or a handler.

def reset_logging():
    manager = logging.root.manager
    manager.disabled = logging.NOTSET
    for logger in manager.loggerDict.values():
        if isinstance(logger, logging.Logger):
            logger.setLevel(logging.NOTSET)
            logger.propagate = True
            logger.disabled = False
            logger.filters.clear()
            handlers = logger.handlers.copy()
            for handler in handlers:
                # Copied from `logging.shutdown`.
                try:
                    handler.acquire()
                    handler.flush()
                    handler.close()
                except (OSError, ValueError):
                    pass
                finally:
                    handler.release()
                logger.removeHandler(handler)

Needless to say, you must setup your logging after running reset_logging().

user_5645090
Related