Schedule Filebeat daemonset on particular ip nodes in kubernetes

utkarsh saraf 0
Viewed 12

I am trying to gather logs of kubernetes cluster into elk cluster via filebeat.

As of now, my filebeat runs on all ec2 nodes as daemon and its working fine but I want to schedule the filebeat on the ip address of the node i choose.

My kubernetes version is 1.15 and helm chart version is 2.17 due to which i am using helm chart of elasticsearch with version 7.17.3.

As per documentation of kubernetes,this can achieved and i have tried to modify helm chart with following entry in filebeat but then no node comes up:

daemonset:
  # Annotations to apply to the daemonset
  annotations: {}
  # additionals labels
  labels: {}
  affinity: 
    nodeAffinity:
     requiredDuringSchedulingIgnoredDuringExecution:
     nodeSelectorTerms:
     - matchFields:
       - key: metadata.name
         operator: In
         values:
         - 10.17.7.7

Kindly help.

filebeat kubernetes kubernetes-helm elasticsearch
1 Answers

Kindly follow the steps as mentioned:

  1. Get the lables of all nodes by running the following command:
kubectl get nodes --show-labels

ip-xx-xx-x-xx.xx.compute.internal    Ready    worker                     511d   v1.15.11   KubernetesCluster=XXdev.com,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=m5.xlarge,beta.kubernetes.io/os=linux,cattle.io/creator=norman,failure-domain.beta.kubernetes.io/region=xx,failure-domain.beta.kubernetes.io/zone=xx,kubernetes.io/arch=amd64,kubernetes.io/hostname=XXdev-aworker1,kubernetes.io/os=linux,node-role.kubernetes.io/worker=true
ip-xx-xx-x-xx.xx.compute.internal   Ready    controlplane,etcd,worker   688d   v1.15.11   KubernetesCluster=XXdev.com,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=m5.2xlarge,beta.kubernetes.io/os=linux,cattle.io/creator=norman,failure-domain.beta.kubernetes.io/region=xx,failure-domain.beta.kubernetes.io/zone=xx,kubernetes.io/arch=amd64,kubernetes.io/hostname=XXdev-all3,kubernetes.io/os=linux,node-role.kubernetes.io/controlplane=true,node-role.kubernetes.io/etcd=true,node-role.kubernetes.io/worker=true
  1. now in the labels column, kindly see the key value pair kubernetes.io/etcd=true and mention the same in above code
daemonset:
  # Annotations to apply to the daemonset
  annotations: {}
  # additionals labels
  labels: {}
  affinity: 
    nodeAffinity:
     requiredDuringSchedulingIgnoredDuringExecution:
     nodeSelectorTerms:
     - matchFields:
       - key: kubernetes.io/etcd
         operator: In
         values:
         - true
  1. Another way to achieve same by nodeSelector is as follows:
nodeSelector:
    kubernetes.io/etcd: true
utkarsh saraf0
Related