I’ve got multiple time series latency metrics and one for each kubernetes pod. I need to calculate the p99 latency metric for the entire service ,i.e, all the pods combined, over every 5min interval. How do I do that?
Here’s the sample time series data:
service_latency(app=“payments”,pod_name=“pod1”)
service_latency(app=“payments”,pod_name=“pod2”)
service_latency(app=“payments”,pod_name=“pod3”)
PromQl that I have so far:
quantile_over_time(0.99, service_latency{app=“payments”}[5m])
However, the above PromQL returns p99 metric for each pod rather than for the entire service. I also am not sure if it is indeed calculating the p99 for every 5 minutes like I intended it to.
Can a PromQL expert here please help!