Why changing the log level dynamically in python does not work

The other answer is "working", but doesn't really tell the whole story here. The answer's claim that you need to change on the logger and its handlers is incorrect.

In fact, there are two setLevel methods (on loggers and handlers). The level set in the logger determines which severity of messages it will pass to its handlers. The level set in each handler determines which messages that handler will send on. In normal usage, setting just one of them up to CRITICAL would have been enough to filter that INFO event.

It doesn't work in your case because the logger is not being created in the documented way:

Note that Loggers should NEVER be instantiated directly, but always through the module-level function logging.getLogger(name).

Creating the logger with logging.Logger(...) has bypassed the logging module's global state, and defeated the hierarchy of the logging tree. Specifically, it has caused an incorrectly cached result to be returned by logger.isEnabledFor().

If you create in the usual way, setting the level on a logger instance will work without fiddling handlers:

import logging

# logger = logging.Logger("MyLogger", level=logging.INFO)  # no
logger = logging.getLogger("MyLogger")  # yes

formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
console = logging.StreamHandler()
console.setFormatter(formatter)
logger.addHandler(console)

logger.setLevel("INFO")
logger.info("should show up")
logger.setLevel(logging.CRITICAL)
logger.info("should not show up")

To change the logging level you need to change it on the parent and its handlers:

level = logging.CRITICAL
logger.setLevel(level)
for handler in logger.handlers:
    handler.setLevel(level)