Datadog - multi-line logs excluding the last line

Viewed 83

I have prepared Datadog agent in a container.

Dockerfile

FROM gcr.io/datadoghq/agent:latest AS datadog
LABEL description="Datadog"

## Volumes
VOLUME ["/var/log/myapp"]

LABEL "com.datadoghq.ad.logs"=' \
[{ \
    "source": "myapp", \
    "service": "testss", \
    "type": "file", \
    "path": "/var/log/myapp/bot0.log", \
    "log_processing_rules": \ 
    [ \
        { \
            "type": "multi_line", \
            "name": "log_start_with_date", \
            "pattern" : "\\d{4}-(0?[1-9]|1[012])-(0?[1-9]|[12][0-9]|3[01]) (0?[0-9]|1[0-9]|2[0-4]):([0-6][0-9]):([0-6][0-9]) \\[(.*?)\\]" \
        } \
    ] \ 
}]'

docker-compose.yml:

version: "3.5"
services:
    datadog_service:
        build: 
            context: .
            target: datadog
        pid: host
        environment:
            - DD_API_KEY=<my_api_key>
            - DD_SITE=datadoghq.eu
            - DD_LOGS_ENABLED=true
            - DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=false
            - DD_CONTAINER_EXCLUDE_LOGS="name:datadog-agent"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - /proc/:/host/proc/:ro
            - /sys/fs/cgroup:/host/sys/fs/cgroup:ro
            - /var/lib/docker/containers:/var/lib/docker/containers:ro

Log entries look like:

2022-08-08 13:43:41 [INFO] (Author)
Log1

2022-08-08 13:53:41 [INFO] (Author)
Log2
Sublog2.1
Sublog2.2

This Datadog setup works fine, except for the last log line.

Last line is not ready - it can still write some data to this block of log, eg. "Sublog2.3".

Datadog will interpret "Sublog2.3" as a new log entry, which is incorrect ...

Any idea how to fix it?

0 Answers
Related