I need to deactivate RepeatedMsgReduction on my Linux server to allow fail2ban to evaluate every failed login attempt. (See this bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=440037)
As I don't want to flood my log files, I would love to have that feature only on those logfiles that fail2ban scans, especially mail.warn.
Is there a way to set a conditional rule/filter in rsyslog.conf that sets
$RepeatedMsgReduction = off
iff the message is bound to mail.warn?
I appreciate the research provided here showing how to selectively disable rsyslog message reduction. However, I think it important to recognize that there are arguments in favor of disabling reduction entirely.
Background:
Syslog's MARK facility is intended to provide evidence that syslog (and by inference the server itself) is running by periodically logging an entry as:
Sep 5 11:41:28 servername rsyslogd: -- MARK --
I came to this SO question because message reduction was defeating the purpose of MARK by aggregating my mark messages:
Sep 5 12:33:46 servername rsyslogd: message repeated 48 times: [-- MARK --]
I.e., the above message is logged in lieu of 48 periodic messages, nullifying their utility as tracking information.
Analysis:
Initially I therefore planned to disable reduction only for MARK messages. However, having read this article I am inclined to turn off RepeatedMsgReduction entirely:
While turning this feature on can save some space in logs, most log analysis tools need to see the repeated messages, they can’t handle the “last message repeated” format.
This is a feature that worked decades ago when logs were small and reviewed by a human, it fails badly on high volume logs processed by tools.
Conclusion:
I recommend simply disabling reduction. Why? Now that disk space is inexpensive and we rely increasingly on automated tools like fail2ban, the benefit of reduction is outweighed by the costs of configuration complexity and information loss.