Kubernetes: Install Fluentd to a namespace only

user_14005291 0
Viewed 2015

I got the fluentd-kubernetes-daemonset charts from https://github.com/fluent/fluentd-kubernetes-daemonset, and deployed fluentd into kube-system namespace as daemonset. It sends entire cluster logs to elasticsearch. We deploy our csc application in the csc namespace. Instead of installing fluentd as daemonset to collect entire cluster logs, we would like to deploy fluentd in the csc namespace only, and only send csc logs (logs in csc namespace) to elasticsearch. Is there a way to do it?

fluentd kubernetes
3 Answers

The link you shared here which has namespace: kube-system so that's why it got created in kube-system namespace. so to use your namespace please edit the yaml file and replace namepsace: csc in all the places of the yaml file and apply it to kubernetes.

and you deployed as daemonsets so it will run fluentd pod on every node.

user_80537140

It`s typical solution for logging architecture to run DaemonSet on every node and collect logs:

Because the logging agent must run on every node, it's common to implement it as either a DaemonSet replica, a manifest pod, or a dedicated native process on the node. However the latter two approaches are deprecated and highly discouraged.

Instead of you may want to shape and parse the log message to your need using parser and filter plugins.

This and this document shows an examples of parsing logs.

user_121865850

You need to fix filter conf in https://github.com/fluent/helm-charts/blob/main/charts/fluentd/values.yaml To collect logs from specific namespace you can just prohibit all others namespaces in section 02_filters.conf:

  <match kubernetes.var.log.containers.**_kube-system_**>
    @type null
    @id ignore_kube_system_logs
  </match>
  ......
  <match kubernetes.var.log.containers.**NAMESPACE**>
    @type null
    @id ignore_NAMESPACE_logs
  </match>

I did this and seems like it's ok

user_60031260
Related