What is the best practice for formatting logs?

user_180295 0
Viewed 59117

I'm writing a piece of honeypot software that will have extensive logging of interactions with it, I plan to log in plaintext .log files.

I have two questions, from someone who isn't too familiar with how servers log.

  1. Firstly how shall I break up my log files, I'm assuming after running this for a month I don't want one big .log file, do I do this by day, month, year? Is there some standard for it?

  2. The format of each line, do I have one standard delimiter that is whatever, *, -, +, anything? Is there a standard anywhere (my googling hasn't brought up much)?

standards logging
8 Answers

Log File

  1. Make Log Entries Meaningful With Context
  2. Use a Standard Date and Time Format
  3. Use Local Time + Offset for Your Timestamps
  4. Use Logging Levels Correctly
  5. Split Your Logging to Different Targets Based on Their Granularity
  6. Include the Stack Trace When Logging an Exception
  7. Include the Name of the Thread When Logging From a Multi-Threaded Application
user_191683830

Today (2022) we love structured logging and log indexers (like Elasticsearch or Loki). So we have to log in NDJson (new line delimetted Json).

Because log delivery agents have difficulty with log file rotation (who loves missing log events??) we avoid rotation (no more logrotate!!): instead we name files using date pattern or auto-incremented sequence and define policy to remove outdated files.

Don't work with individual files! Use centralized log search engines!!

https://en.wikipedia.org/wiki/Common_Log_Format is in the past.

user_1731490
Related