I have been trying to set up Elastic Beanstalk so that all instances across multiple availability zones appear externally to have the same static IP address. To do that, I followed the instructions here:
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-static-IP-address/
I also followed the prerequisites here:
https://aws.amazon.com/premiumsupport/knowledge-center/nat-gateway-vpc-private-subnet/
I created a demo environment (PHP) and installed the sample application - just for a proof of concept. Everything works fine as long as I use only a single availability zone (load balancer is in a public subnet and there are two other private subnets for each instance - but all are in the same availability zone). The moment I try to modify Elastic Beanstalk so that some of the instances are in a different availability zone, I have mixed results. Sometimes I am able to get the application to load in my browser, and other times it just times out. I have tried various combinations of enabling / disabling cross zone load balancing, and I get the same results either way.
When I look at the load balancer, all instances show "In Service", and Elastic Beanstalk reports a status of "OK". When I run the VPC Reachability analyzer, it shows that each instances is reachable from the Internet Gateway.
Here is the configuration I have set up:
Availability Zone us-east-1c
- Subnet 1 (public): This subnet routes traffic to the Internet Gateway. This is the subnet where the load balancer resides.
- Subnet 2 (private): This subnet does not route anything to the Internet Gateway. Instead, it routes all outbound traffic to the NAT (which also resides in the public subnet).
- Subnet 3 (private): This subnet is identical to Subnet 2 (uses the same routing table).
Availability Zone us-east-1b
- Subnet 4 (private): This subnet is identical to Subnet 2 (uses the same routing table).
Availability Zone us-east-1d
- Subnet 5 (private): This subnet is identical to Subnet 2.
So what I am seeing is that as long as I deploy instances only to subnets 2 and 3, everything works fine. As soon as I try to add instances in subnets 4 or 5, that's when the application starts becoming intermittent. Sometimes it works and sometimes it doesn't.
I should also point out that I am trying to load this application in my browser by referencing the URL of the load balancer, and also the auto-generated Elastic Beanstalk URL for the environment. The behavior is the same in both cases.
What am I doing wrong?