Tail docker logs to see recent records, not all

user_6719887 0
Viewed 147224

If you use the Coreutils tail command in Linux, you have a -f option that lets you follow a log file from the log's current position (it does not go to the very beginning of the file and display everything).

Is this functionality available in docker logs without waiting for it to traverse the whole log?

I have tried:

docker logs --since 1m somecontainer

and

docker logs -f --since 1m somecontainer

It appears that it actually traverses the entire log file (which can take a long time) and then starts echoing to the screen once it reaches the time frame you specify.

Is there a way to start tailing from the current point without waiting? Is my best option to always log out to some external file and tail that with the Coreutils tail command?

docker logging
7 Answers

Please read docker logs --help for help. Try below, starting from the last 10 lines. More details here.

docker logs -f --tail 10 container_name
user_84389990

Alternatively, we can check the log by time (eg. since last 2mins) as:

docker logs --since=2m <container_id> // since last 2 minutes
docker logs --since=1h <container_id> // since last 1 hour
user_18454260

use the --tail switch:

>  docker logs -f <container name> --tail 10

this will show the log starting from the last 10 lines onwards

user_44122030

I think your doing it correct and it seem to work as expected when i try it. Are you using some non-default log driver etc?

To follow only new log files you can use -f --since 0m.

user_566860

The default setting for the log driver is a json file format, and the only way I can think of to reliably parse that involves parsing the file from the beginning, which I suspect is exactly what docker does. So the I'm not sure there's an option to do exactly what you are asking. However, there are two log options you can adjust when starting a container with the default json log driver.

  1. max-size: this limits how large a single json logfile will grow to. After this, docker will create a new file. By default it is unlimited (-1).
  2. max-file: this limits the number of json logfiles that will be created up to the max size set above. By default it set to 1.

You can read about these options here: https://docs.docker.com/config/containers/logging/json-file/

I typically set these options with new default values for all containers being run on the docker host using the following lines inside my /etc/docker/daemon.json file:

{
"log-driver": "json-file",
"log-opts": {"max-size": "10m", "max-file": "3"}
}

Those two options say to keep up to 3 different 10 meg json log files. The result is a limit between 20-30 megs of logs per container. You need to trigger a reload on the dockerd process to load this file (killall -HUP dockerd or systemctl reload docker).

You can override this on an individual container by passing the log options on your run command (or inside the compose file):

docker container run --log-opt max-size=5m --log-opt max-file=2 ...

There does not appear to be a way to change the logging options of an existing container, so you will need to recreate your containers to apply these changes.

The end result is that docker may still have to parse the entire file to show you the most recent logs, but the file will be much smaller with automatically rotating logs than the default unlimited logging option.

user_5962850

If you want to get the logs based on the service name (case of docker-compose) you can use this shorthand (nginx here is an example of a service name):

docker logs -f --since=1m $(docker ps -f name=nginx --quiet)
user_67136410

Please read docker logs --help for help but it will help to check the current logs.

docker logs -f container_name/id

user_56625590
Related