Logging date format misrepresents │as ¦

user_2575743 0
Viewed 62

I have a logging script with set formatting. It is using utf-8 symbols for readability, the encoding is set for saving of the files, as seen below.

log = logging.getLogger('general')
fmt = logging.Formatter('%(asctime)s.%(msecs)03d  │  %(levelname)-10s│  %(lineno)4s %(funcName)-20s│  %(message)s',
                        datefmt='%Y/%m/%d │ %H:%M:%S')
handler_error = logging.handlers.RotatingFileHandler("error.log", 'a', encoding="utf-8")
handler_error.setLevel(logging.INFO)
handler_error.setFormatter(fmt)
log.addHandler(handler_error)

Theoretically, all the separators should be │'s, but the one in datefmt it is being changed into ¦. Thus, the resulting log looks like below - notice all the vertical lines added properly except the one set between date and time.

I have tried using unicode \u2502 instead of │, did not help. Any ideas on how to fix this?

screenshot of the log

Edit - an error log from another user's execution.

  File "C:\Users\xxx\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1025, in emit
    msg = self.format(record)
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 869, in format
    return fmt.format(record)
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 610, in format
    record.asctime = self.formatTime(record, self.datefmt)
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 548, in formatTime
    s = time.strftime(datefmt, ct)
UnicodeEncodeError: 'locale' codec can't encode character '\u2502' in position 9: encoding error
utf-8 logging python
1 Answers

Solution: You need to set LC_CTYPE to UTF-8

The following example for more Box Drawings characters shows that there goes some character translation in datefmt string (similar to the difference between Windows commands tree vs tree /A).

# -*- coding: utf-8 -*-

import time, locale
t = time.localtime()
datefmt = '%Y/%m/%d ─│┌┐└┘├┤┬┴┼═║╔╗╚╝╠╣╦╩╬ %H:%M:%S'
print( '\n datefmt:', datefmt)

print( '\nLC_CTYPE:', locale.getlocale(category=locale.LC_CTYPE) )
print( 'strftime:', time.strftime(datefmt, t) )

locale.setlocale(locale.LC_CTYPE, (locale.getlocale()[0], '1252'))
print( '\nLC_CTYPE:', locale.getlocale(category=locale.LC_CTYPE) )
print( 'strftime:', time.strftime(datefmt, t) )

locale.setlocale(locale.LC_CTYPE, (locale.getlocale()[0], 'utf8'))
print( '\nLC_CTYPE:', locale.getlocale(category=locale.LC_CTYPE) )
print( 'strftime:', time.strftime(datefmt, t) )

Output: .\SO\63472673.py

 datefmt: %Y/%m/%d ─│┌┐└┘├┤┬┴┼═║╔╗╚╝╠╣╦╩╬ %H:%M:%S

LC_CTYPE: ('Czech_Czechia', '1250')
strftime: 2020/08/20 ¦--¬L-++T++=¦-¬L-¦¦T¦+ 19:29:14

LC_CTYPE: ('Czech_Czechia', '1252')
strftime: 2020/08/20 -¦+++++¦--+-¦++++¦¦--+ 19:29:14

LC_CTYPE: ('Czech_Czechia', 'utf8')
strftime: 2020/08/20 ─│┌┐└┘├┤┬┴┼═║╔╗╚╝╠╣╦╩╬ 19:29:14

Unfortunately, I can't find that character translation in Python source code (one can see that there is a bug with (my native) code page 1250: characters ─ and │ are translated as ¦ and -, respectively i.e. in the reverse order…

user_34394040
Related