format shell script logs and write to a file

The best choice to reliably translate a shell string into a valid JSON string is to use a JSON parser/formatter. The most popular one is jq.

Here is an implementation of a timestamped JSON message logger using jq:

#!/usr/bin/env bash

json_logger() {
  while read -r msg || [ -n "$msg" ]; do
    jq \
      -nc \
      --arg msg "$msg" \
      '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": $msg }'
  done
}


----------

EDIT:

As KamilCuk wrote:

We can do better without slow bash loop - it's just jq --raw-input '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": . }'

So here is an improved json_logger:

json_logger() {
  jq \
    --raw-input \
    --compact-output \
    '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": . }'
}

Addendum:

Harshit Kushwaha wrote:

For example, I need to print the method name in json from where the json_logger was called. How can I modify the json_logger and how to use it then in echo?

Here is an implementation with adding the method name if provided as an argument to the json_logger function:

#!/usr/bin/env bash

IFS= read -r -d '' __JQ_LOGGER_SCRIPT <<'JQSCRIPT'
{
  "timestamp": now | strftime("%Y-%m-%d %H:%M:%S"),
  "message": .
} |
  if ($name | length) != 0
  then
    . + { "method": $name }
  else
    .
  end
JQSCRIPT

json_logger() {
  jq \
    --raw-input \
    --compact-output \
    --arg name "$1" \
    "$__JQ_LOGGER_SCRIPT"
}

echo "I want to be a json message!" | json_logger
echo "I also want to be a json message!" | json_logger echo

printf %s $'I am a message with "quoted text" and some special characters: \'\t\7\42\\\'; that can only be properly converted to JSON with a JSON formatter and parser.' | json_logger printf

Produces this JSON outputs:

{"timestamp":"2021-01-29 14:02:46","message":"I want to be a json message!"}
{"timestamp":"2021-01-29 14:02:46","message":"I also want to be a json message!","method":"echo"}
{"timestamp":"2021-01-29 14:02:46","message":"I am a message with \"quoted text\" and some special characters: '\t\u0007\"\\'; that can only be properly converted to JSON with a JSON formatter and parser.","method":"printf"}

Thanks for the solutions.

I was able to write logs from script executions to the log file with below implementation:

1. Creating a json logger

     json_logger() {
            # Values for json
            current_timestamp="$(date +\"%Y-%m-%dT%H:%M:%S\")"
            param1="$param1value"
            param2="$param2value"
            log_level=$1
            script_name=$(basename $2 .sh)
   
            # Create json and re-direct to log file
            jq --raw-input --compact-output \
            '{
                    "@timestamp": '$current_timestamp',
                    "parameter1": "'$param1'",
                    "parameter2": "'$param2'",
                    "log-level": "'$log_level'",
                    "logger": "'$script_name'",
                    "message": .
            }' >> /var/log/app.log 2>&1 
        }
   

2. And then executing scripts as below

      "script.sh" 2>&1 | json_logger "INFO" {job_name or script_name}
   

./script.sh | awk -v squot="'" '{ print "{\"timestamp\": \""strftime("%Y-%m-%d %H:%M:%S")"\", \"message\": \""$0"\"}" }' mess | jq >> jsonmessages.log

Using GNU awk, print the timestamp in the required format using awk's strftime function, along with double quotes and the other data in the format required. Pipe through to jq to ensure valid json