Amazon EMR - missing ingress rule from EmrManagedMasterSecurityGroup on port 9443

user_1770601 0
Viewed 1602

I am trying to get my head around this, but cannot figure it out why

  1. I have an EMR cluster to be deployed in AWS Private Subnet.
  2. I checked the documentation here.
  3. From the above I understood the following:

a. For my EMR custom security group, I DO NOT NEED to specify ingress rule for port 9443 i.e. it can be the same as the ElasticMapReduce-Master-Private. The following are my inbound/outbound rules for the EMR Managed Security Group for Master/Slave instances (NOT Service)

Outboundenter image description here

Inboundenter image description here

b. For my custom service security group I can keep it the same as ElasticMapReduce-ServiceAccess, I need to specify ingress rule for port 9443 like the following (Terraform):

resource "aws_security_group_rule" "allow_tcp_from_master_to_service" {
  type                     = "ingress"
  from_port                = 9443
  to_port                  = 9443
  protocol                 = "tcp"
  security_group_id        = join("", aws_security_group.ml.*.id)
  source_security_group_id = join("", aws_security_group.ml_emr.*.id)

  lifecycle {
    create_before_destroy = false
  }
}

But when I deploy this using terraform, I get the AWS error:

Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATED_WITH_ERRORS: VALIDATION_ERROR: ServiceAccessSecurityGroup is missing ingress rule from EmrManagedMasterSecurityGroup on port 9443

I am struggling to understand why AWS is saying I need this when I don't? From the documentation it's not clear where this mentioned, so I would appreciate if someone could clarify what's expected here.

Regards,

aws-security-group amazon-emr
2 Answers

Did you have the same SG for master and task instances?

I had different SGs for master and task instances and I solved it by using the same SG for both

user_121432950
user_22349790
Related