I have a gauge metric badness which goes up when my service is performing poorly. There is one gauge per instance of the service and I have many instances.
I can take a max over all instances so that I can see how bad the worst instance is:
max(badness)
This graph is noisy because the identity of the worst instance, and how bad it is, changes frequently. I would like to smooth it out by applying a moving average. However, this doesn't work (I get a PromQL syntax error):
avg_over_time(max(badness)[1m])
How can I apply avg_over_time() to a timeseries that has already been aggregated with the max() operator?
My backend is VictoriaMetrics so I can use either MetricsQL or pure PromQL.