How to parse logs and also stream logs to the docker demon?

user_11958101 0
Viewed 28

I am working on building a monitoring system for our NGINX Plus servers, we are running these on Kubernetes. By default, NGINX Plus comes with API to get stats but they don't go into specific codes they get aggregated like 1XX, 2xx I am looking to get metrics for specific codes like 200, 301, 302, 404, and the only way I could find is to parse NGINX log files using tools like telegraf, but the problem is by default NGINX stream logs using STDOUT and STDERR, and telegraf cant really capture logs from streams and storing logs on the container doesn't make sense as it will not get picked by fluent-bit.

I am running NGINX and telegraf in the same pod, K8s has shareProcessNamespace which allows accessing container file systems in the same pod but NGINX is not really writing these logs to file its streaming.

is there any way I can parse logs and also stream logs to the docker demon?

kubernetes docker nginx
1 Answers

by default NGINX stream logs using STDOUT and STDERR

So you should deploy an NGiNX configured to log to access.log or syslog

And keep in mind the telegraf NGiNX input plugin might not have the details you need:

Using this configuration:

[[inputs.nginx]]
 ## An array of Nginx stub_status URI to gather stats.
 urls = ["http://localhost/status"]

When run with:

./telegraf --config telegraf.conf --input-filter nginx --test

It produces:

* Plugin: nginx, Collection 1
> nginx,port=80,server=localhost >accepts=605i,active=2i,handled=605i,reading=0i,requests=12132i,waiting=1i,writ>ing=1i 1456690994701784331

(unless you configure Metrics Collection, in conjunction with Prometheus)

VonC_63090
Related