Ideolog (PyCharm): how to configure log format for standard logging library

I just had the same issue.

I used the following config:

My logging format looks like this:

"%(asctime)s %(name)-30s %(levelname)-8s %(message)s"

Log Parsing Pattern:

pattern="^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})\s(\S*)\s*(\w*)\s*(.*)$" timePattern="yyyy-MM-dd HH:mm:ss,SSS" linePattern="^\d" timeId="0" severityId="2" fullmatch="true"

Log Hightlight Patterns:

pattern="^\s*ERROR?\s*$" action="HIGHLIGHT_LINE" fg="-65536" stripe="true"
pattern="^\s*WARNING?\s*$" action="HIGHLIGHT_LINE" fg="-22016" bold="true"
pattern="^\s*INFO\s*$" action="HIGHLIGHT_LINE" fg="-12599489"
pattern="^\s*DEBUG\s*$" action="HIGHLIGHT_LINE" fg="-14927361" stripe="true"
pattern="^\s*CRITICAL\s*$" action="HIGHLIGHT_LINE" fg="-65536" bold="true" italic="true" stripe="true"

HTH and YMMV...

if you don't want to struggle much with regular expressions you can do an easy one

(INFO)

will match "INFO" and you can do the rest for the other levels for a quick visualization.

Had an issue on a very short test file - only four entries - and was searching forever why my correct pattern was not applied.

As the official docs state here https://github.com/JetBrains/ideolog/wiki/Custom-Log-Formats

Selecting the right pattern

"In order to detect the format of a log file, all existing patterns are matched against first 25 lines of a file. The one with most matches gets selected, if the amount of matches is above 5. Otherwise, a dumb per-line parser is used."

So if your log file is below 5 entries, the algorithm says it can not detect the correct pattern - and does not apply any pattern, even if there is only one active pattern and the regex works 100% fine.

Groups

"In addition, you must specify capture group indices for time, severity and category. Capture groups are numbered starting with 1. If you don't have a capture group for an item, specify 0."

Hence: Group Numbering starts at 1 !

My config

str_logging_format = "%(asctime)s - %(name)-12s - %(levelname)-8s - %(message)s"

# -- Ideolog Config --
# :pattern:
# ^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})\s-\s(\S*)\s*-\s(\w*)\s*-\s(.*)$
# :timePattern: yyyy-MM-dd HH:mm:ss,SSS
# :linePattern: ^\d
# :time Group : 1
# :severity Group : 3
# :Category Group: 2
#
# :highlighting:
# pattern="^\s*CRITICAL\s*$" action="HIGHLIGHT_LINE" fg="FF0000" bold="true" stripe="true"
# pattern="^\s*ERROR\s*$" action="HIGHLIGHT_LINE" fg="FF5F62" stripe="true"
# pattern="^\s*WARNING\s*$" action="HIGHLIGHT_LINE" fg="C73EC8" bold="true"
# pattern="^\s*INFO\s*$" action="HIGHLIGHT_LINE" fg="08C8EA"
# pattern="^\s*DEBUG\s*$" action="HIGHLIGHT_LINE" fg="3940C8"

PS: https://regex101.com/ is awesome to test your pattern.

THis is for main.py

import logging as log
logger = r'logger.log'


if __name__ == "__main__":
log.basicConfig(filename=logger,
                format='%(asctime)s - %(threadName)s - %(name)s - %(levelname)s - %(message)s',
                datefmt='%Y-%m-%d %H:%M:%S', level=log.DEBUG)

and this is for highlighting (idealog)

message pattern:   ^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})\s(\S*)\s*(\w*)\s*(.*)$

message start pattern:    ^\d

time format         HH:mm:ss,SSS
time capture group:       0
severity capt group:      3
cat capt group:           1

I do that and all works. Good luck

Based on Kirill's answer I finally got it working in pycharm too. In case helpful, here is my setup.

My log file formatter is

fmtter = logging.Formatter(fmt="%(asctime)s-%(levelname)s-%(message)s")

Typical lines of one of my log files look like this

2020-11-09 14:26:08,567-INFO-measuring Op/frame for CNN
2020-11-09 14:26:09,045-INFO-Op/frame: 249.64M

My message pattern is

^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})-(\S*)-(.*)$

Note the explicit matching of entire date/time string, then '-', then level string, another '-', then rest of the line is the payload. The \S matches the level name with no spaces. The boost regexp page at https://www.boost.org/doc/libs/1_44_0/libs/regex/doc/html/boost_regex/syntax/perl_syntax.html is helpful and I used sublime to test the expression.

Time format is yyyy-MM-dd HH:mm:ss,SSS
Time is group 0
Severity group 1
Category group 2