Trigger alert based on an additional condition of sample count

Viewed 54

We have set up a monitor which notifies us of potential performance issues for each tenant in our system.

However, there are days where activity might be very low on some specific tenant, and the request count may not be worthy enough of triggering an alert.

For example, the monitor should only trigger on BOTH of the following:

  • if requests' duration p50 over 24 hours exceed 100ms, AND
  • if request count is greater than 100 in 24 hours.

How can I achieve the request count condition? There seems to be no way to do this out of the box, but is there a creative way to go about it?

Threshold alert datadog

1 Answers

I think composite monitors will handle this case. You'd create two monitors, one for the request duration and one for the request count, and then set the trigger condition for the composite monitor to be &&.

composite monitor screenshot from datadog docs

Related