Suppressing debug messages from neptune library

user_308827 0
Viewed 111

Is there a way to suppress the DEBUG messages outputted by the NEPTUNE library? https://github.com/neptune-ai/neptune-client

This is what I am doing currently:

tracker = neptune.init(project=f'WORKSPACE/PROJECT_NAME',
                       capture_hardware_metrics=False,
                       source_files=[],
                       capture_stdout=False,
                       capture_stderr=False)

These are the DEBUG messages I get:

2022-05-03  11:31:58 DEBUG https://app.neptune.ai:443 "POST /api/leaderboard/v1/attributes/ping?experimentId=aaa HTTP/1.1" 200 0
2022-05-03  11:32:08 DEBUG ping({'experimentId': 'aaa', '_request_options': {'timeout': 10, 'connect_timeout': 10}})
python
3 Answers

Option 1

It seems counterintuitive, but passing mode=debug mode to Neptune will surpress Neptune related DEBUG messages:

tracker = neptune.init(
    project=f'WORKSPACE/PROJECT_NAME',
    mode="debug",
    capture_hardware_metrics=False,
    source_files=[],
    capture_stdout=False,
    capture_stderr=False
)

Another way to enable this functionality is to set environment variable CONNECTION_MODE.

Option 2

If you want to be able to fine tune which messages are logged to the Neptune website/server, consider using the NeptuneHandler provided by the Neptune client.

Example:

>>> import logging
>>> import neptune.new as neptune
>>> from neptune.new.integrations.python_logger import NeptuneHandler
>>> logger = logging.getLogger("root_experiment")
>>> logger.setLevel(logging.INFO)
>>> run = neptune.init(project="neptune/sandbox")
>>> npt_handler = NeptuneHandler(run=run) # you can also pass `level=logging.INFO` here
>>> logger.addHandler(npt_handler)
>>> logger.info("Starting data preparation")
...
>>> logger.info("Data preparation done")

The above example sends all logs above log levels of severity > DEBUG to the Neptune website/server. For reference, log level severity for python are as follows (CRITICAL > ERROR > WARNING > INFO > DEBUG). I've included Python's documentation on this topic in the references below.

References

https://docs.neptune.ai/you-should-know/connection-modes#debug

https://docs.neptune.ai/api-reference/integrations/python-logger

https://github.com/neptune-ai/neptune-client/blob/8950d174336ce689480607870119a0b276ebbacf/neptune/new/integrations/python_logger.py

https://docs.python.org/3/howto/logging.html#when-to-use-logging

user_8068760

You can completely disable stdout. Use the following functions:

import sys, os

# Disable
def disable_stdout():
    sys.stdout = open(os.devnull, 'w')

# Restore
def enable_stdout():
    sys.stdout = sys.__stdout__

Example:

disable_stdout()

<Code to Run without logging/printing>

enable_stdout()

WARNING:
This is a quite dangerous approach, as it will probably hide other important/usefull messages.
Use it as a last resort

user_125932960

You could use the logging module.

Simply, import the module and get the logger instance by using logging.getLogger, and then use the setLevel function, pass in the level parameter as the level above the types of warnings you want to show(i.e. the type of problem before which you want to ignore).

Since these are debug messages, you could set a level like critical by using logging.CRITICAL, this would cause the debug messages from that library to be ignored.

Read https://docs.python.org/3/library/logging.html#logging.Logger.setLevel for more information

user_143344400
Related