I am trying to discover both pods and nodes on the same job using kubernetes_sd_configs and use their labels together. I have blackbox-exporter on multiple pods in my cluster on different nodes, my goal is to monitor each of them, but I am having trouble with the metric relabelling. I am trying to achieve something like this:
http://<pod-ip1>:8082/metrics?instance=<node-ip1>
http://<pod-ip2>:8082/metrics?instance=<node-ip1>
http://<pod-ipN>:8082/metrics?instance=<node-ip1>
.
.
http://<pod-ip1>:8082/metrics?instance=<node-ip2>
http://<pod-ip2>:8082/metrics?instance=<node-ip2>
http://<pod-ipN>:8082/metrics?instance=<node-ip2>
.
.
My current configuration looks like the following, but the pod URL is missing:
- job_name: 'kubernetes-pods'
metrics_path: /probe
params:
module: [ping]
kubernetes_sd_configs:
- role: node
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: <pod_should_be_here>:9115
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __param_target
- source_labels: [__param_target]
target_label: instance