I am parsing URLs in Python using tldextract library. When I run it, it generates many FILELOCK messages that I want to suppress. I do not want to suppress any other stdout messages (I have many logging messages - informational that I want to see). I am using multiprocessing and using all 8 cores of my laptop to run on a large set of URLs.
Any suggestions? Here are the details:
Example code is below. I tried using contextlib (https://docs.python.org/3/library/contextlib.html#contextlib.redirect_stdout). But, maybe I am not using it properly.
import io, tldextract, contextlib, os
def get_sld(self, reg_domain_name):
"""
Get only the second-level domain.
:param reg_domain_name: sports.cnn.com
:return: cnn
"""
domain = reg_domain_name
#f = io.StringIO()
with contextlib.redirect_stdout(os.devnull):
if reg_domain_name:
try:
ext = tldextract.extract(reg_domain_name)
except:
logger.info(f'Unable to extract domain from {reg_domain_name}. Using given fqdn.')
else:
domain = ext.domain
return domain
The above code generates many messages like the below..which I want to suppress:
2020-10-29 20:00:11,750 - filelock - filelock - INFO - Lock 4992537872 released on /Users/sri/p37/lib/python3.7/site-packages/tldextract/.suffix_cache/urls/62bf135d1c2f3d4db4228b9ecaf507a2.tldextract.json.lock
2020-10-29 20:00:11,765 - filelock - filelock - INFO - Lock 4992398992 released on /Users/sri/p37/lib/python3.7/site-packages/tldextract/.suffix_cache/publicsuffix.org-tlds/de84b5ca2167d4c83e38fb162f2e8738.tldextract.json.lock
2020-10-29 20:00:11,790 - filelock - filelock - INFO - Lock 4992506640 acquired on /Users/sri/p37/lib/python3.7/site-packages/tldextract/.suffix_cache/publicsuffix.org-tlds/de84b5ca2167d4c83e38fb162f2e8738.tldextract.json.lock
2020-10-29 20:00:11,796 - filelock - filelock - INFO - Lock 4992529872 acquired on /Users/sri/p37/lib/python3.7/site-packages/tldextract/.suffix_cache/urls/62bf135d1c2f3d4db4228b9ecaf507a2.tldextract.json.lock
2020-10-29 20:00:11,802 - filelock - filelock - INFO - Lock 4992529872 released on /Users/sri/p37/lib/python3.7/site-packages/tldextract/.suffix_cache/urls/62bf135d1c2f3d4db4228b9ecaf507a2.tldextract.json.lock