How can I override "source" in DataDog when running docker containers?

user_2931852 0
Viewed 142

I have a docker-compose file where I run several containers, including DataDog agent. Everything is working fine, except the "source" that DataDog picks when sending container logs to DataDog.

There, for example, if I use the official Nginx image, since the docker image is called "nginx", the datadog source is picked by default as "nginx", using the Nginx DataDog pipelines to parse the logs.

But, if I am using a docker image called "my-account/hello-world", then the DataDog picks by default the source as "hello-world". If my image is inspired by Nginx, and reporting the logs in an "nginx format", I would like to be able to change the DataDog source to "nginx", so DataDog can pick up by default the Nginx logs pipeline.

How can I achieve this?

I have tried to set up a DD_SOURCE environment variable, but no luck, DataDog is still picking the image name as "source". I have tried also to set up the source as tag (DD_TAGS=source:nginx), but DataDog does not pick it up.

Thanks!

datadog docker-compose
1 Answers

I had a similar problem which I finally solved. I think you can adapt this solution to your case, too.

The docs state that you should use this format:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    ad.datadoghq.com/<CONTAINER_IDENTIFIER>.logs: '[<LOG_CONFIG>]'
spec:
  containers:
    - name: '<CONTAINER_IDENTIFIER>'

Since nginx-ingress-controller defaults to the container name controller I used that as <CONTAINER_IDENTIFIER>:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    ad.datadoghq.com/controller.logs: '[<LOG_CONFIG>]'
spec:
  containers:
    - name: 'controller'

My working final configuration including the <LOG_CONFIG> and using a Deployment instead of a Pod, to be more realistic, looks like this:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    metadata:
      annotations:
        ad.datadoghq.com/controller.logs: |
          [{"service": "controller", "source": "nginx-ingress-controller"}]
    spec:
      containers:
        - name: controller

It might be worth noting that without the array braces [] around the log config, logs will simply not be ingested at all.

Datadog now automatically knows how to parse the logs (it said controller before):

Datadog parsing logs correcty

user_53282240
Related