Using regex in gcp monitoring filter

Viewed 920

I am trying to monitor an instance group and have a regex based filter. The Query looks like this

fetch instance_group
| metric 'compute.googleapis.com/instance_group/size'
| filter (resource.instance_group_name =~ 'nat-ecmp-[1|2]')
| group_by 1m, [value_size_mean: mean(value.size)]
| every 1m

Now i am trying to do the same in terraform.

This is what I have come up till now

condition_threshold {
      filter   = <<-EOF
                resource.type=\"instance_group\"
                metric.type=\"compute.googleapis.com/instance_group/size\"
                resource.label."instance_group_name"=~"nat-ecmp-[1|2]"
               EOF
      duration        = "300s"
      comparison      = "COMPARISON_LT"
      threshold_value = "1"

But this yields the error

Error creating AlertPolicy: googleapi: Error 400: Field alert_policy.conditions[0].condition_threshold.filter had an invalid value of "resource.type="instance_group"
metric.type="compute.googleapis.com/instance_group/size"
resource.label."instance_group_name"=~"nat-ecmp-[1|2]"
": Could not parse filter "resource.type=\"instance_group\"\nmetric.type=\"compute.googleapis.com/instance_group/size\"\nresource.label.\"instance_group_name\"=~\"nat-ecmp-[1|2]\"\n"; syntax error at line 3, column 37, token '=~'

I have also tried this

    condition_threshold {
      filter   = <<-EOF
                resource.type=\"instance_group\"
                metric.type=\"compute.googleapis.com/instance_group/size\"
                resource.label.\"instance_group_name\"=~\"nat-ecmp-[1|2]\"
               EOF
      duration        = "300s"
      comparison      = "COMPARISON_LT"

But I get the same result. What have I missed here ?

1 Answers

I found my solution after reading multiple times the Google documentation here.

You need to use:

metric.labels.instance_name = monitoring.regex.full_match("^gke-(hipster|nginx).*")

The full example for my use case is:

resource "google_monitoring_alert_policy" "alert_policy" {
  display_name = "Dataflow fail alert policy"
  combiner     = "AND"
  conditions {
    display_name = "test condition"
    condition_threshold {
      trigger {
        percent = null
        count   = 1
      }
      aggregations {
        per_series_aligner   = "ALIGN_MAX"
        group_by_fields      = []
        alignment_period     = "3600s"
        cross_series_reducer = "REDUCE_NONE"
      }
      threshold_value      = 0
      duration             = "300s"
      comparison           = "COMPARISON_GT"
      filter               = "metric.type=\"logging.googleapis.com/user/dev_dataproc_job_error\" AND resource.type=\"cloud_dataproc_job\" AND resource.label.job_id!=monitoring.regex.full_match(\"foo_bar.*\")"
    }
  }
}
Related