Adding google cloud armor to Terraform gke and kubernetes

Question

I am trying to add google cloud armor to my Terraform project that deploys app using Kubernetes. I follow this example. But, in my case, I want to create this rules instead: https://github.com/hashicorp/terraform-provider-google/blob/master/examples/cloud-armor/main.tf

Close all traffics for all IPs on all ports but open traffic for all IPs on port 80 and 443

Then I added a file also called web_application_firewall.tf under the directory terraform/kubernetes with the following configuration:

# Cloud Armor Security policies
resource "google_compute_security_policy" "web-app-firewall" {
  name        = "armor-security-policy"
  description = "Web application security policy to close all traffics for all IPs on all ports but open traffic for all IPs on port 80 and 443"

  # Reject all traffics for all IPs on all ports
  rule {
    description = "Default rule, higher priority overrides it"

    action   = "deny(403)"
    priority = "2147483647"

    match {
      versioned_expr = "SRC_IPS_V1"

      config {
        src_ip_ranges = ["*"]
      }
    }

    
  }

  # Open traffic for all IPs on port 80 and 443
  #rule {
  #  description = "allow traffic for all IPs on port 80 and 443"

  #  action   = "allow"
  #  priority = "1000"

  #  match {
  #    versioned_expr = "SRC_IPS_V1"

  #    config {
  #      src_ip_ranges = ["*"]
  #    }
  #  }
  #}
}

resource "google_compute_firewall" "firewall-allow-ports" {
  name    = "firewall-allow-ports"
  network = google_compute_network.default.name

  allow {
    protocol = "icmp"
  }

  allow {
    protocol = "tcp"
    ports    = ["80"]
  }

  source_tags = ["web"]
}

resource "google_compute_network" "default" {
  name = "test-network"
}

Here, I deactivate port 445 but after I redeployed, I still have an access to the web app. Could you please let me know what I did wrong here? Thank you in advance.

user_11148139 · Answer

First of all I would like to clarify a few things.

Cloud Armor

Google Cloud Armor provides protection only to applications running behind an external load balancer, and several features are only available for external HTTP(S) load balancer.

In short, it can filter IP addresses but cannot block ports, it's firewall role.

In question you have deny rule for all IP's and allow rule (which is commented), however both rules have src_ip_ranges = ["*"] which applies to all IPs which is a bit pointless.

Terraform snippet.

I have tried to apply terraform-provider-google with your changes, however I am not sure if this is exactly what you have. If you could post your whole code it would be more helpful to replicate this whole scenario as you have.

As I mentioned previously, to block ports you need to use Firewall Rule. Firewall Rule applies to a specific VPC network, not all. When I tried to replicate your issue I found that you:

Create new VPC network

resource "google_compute_network" "default" {
  name = "test-network"
}

Created Firewall rule

resource "google_compute_firewall" "firewall-allow-ports" {
  name    = "firewall-allow-ports"
  network = google_compute_network.default.name

  allow {
    protocol = "icmp"
  }

  allow {
    protocol = "tcp"
    ports    = ["80"]
  }

  source_tags = ["web"]
}

But where did you create VMs? If you followed github code, your VM has been created in default VPC:

  network_interface {
    network = "default" ### this line
    access_config {
      # Ephemeral IP
    }

In Terraform doc you can find information that this value indicates to which network the VM will be attached.

network_interface - (Required) Networks to attach to the instance. This can be specified multiple times.

Issue Summary

So in short, you have created new VPC (test-network), Created VPC rule ("firewall-allow-ports") to allow only ICMP protocol and TCP protocol on port 80 with source_tags = web for new VPC - test-network but your VM has been created in default VPC which might have different firewall rules to allow whole traffic, allow traffic on port 445 or many more variations.

Possible solution

Using default as the name of resource in terraform might be dangerous/tricky as it can create resources in different locations than you want. I have changed this code a bit to create a VPC network - test-network, use it for firewall rules and in resource "google_compute_instance".

resource "google_compute_network" "test-network" {
  name = "test-network"
}

resource "google_compute_firewall" "firewall-allow-ports" {
  name    = "firewall-allow-ports"
  network = google_compute_network.test-network.name

  allow {
    protocol = "icmp"
  }

  allow {
    protocol = "tcp"
    ports    = ["80", "443"] ### before was only 80
  }

  source_tags = ["web"]
}

resource "google_compute_instance" "cluster1" {
  name         = "armor-gce-333" ### previous VM name was "armor-gce-222"
  machine_type = "f1-micro"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
    }
  }

  network_interface {
    network = "test-network"
    access_config {

...

As you can see on the screens below, it created Firewall rule also for port 443 and in VPC test-network you can see VM "armor-gce-333".

Summary Your main issue was related that you have configured new VPC with firewall rules, but your instance was probably created in another VPC network which allowed traffic on port 445.

Adding google cloud armor to Terraform gke and kubernetes

1 Answers