I'm trying to create highly available applications using the AWS ECS infrastructure in front of an Application Load Balancer.
I have used templates from this repo (supported by AWS) https://github.com/widdix/aws-cf-templates
specifically the ecs/cluster combined with my services using ecs/service-cluster-alb.yaml
This sets up an ALB in front of a cluster, with CPU/memory/500 alarms to grow/shrink the cluster size and individual services as and when the capacity requires.
I've done some experimenting with loader.io and found the bottleneck to be roughly 3000 request per minute. which is pretty poor.
In the following test I tried 5000 in 1 minute;
- The service metrics for cpu and memory barely went above 5%
- container metrics basically stayed the same
- ALB Capacity units went to 0.27 < !!!! (didnt even hit 1Cu)
The test only managed to get 600+ successful requests followed by 200 timeouts that stopped the tests.
What is the solution to handling more traffic than this?
I also did this my instance type ar various t2 / t3 instances and even a zmetal instance and it made no difference at all. It seems the bottle neck is 100% elsewhere
The end point I tested was very basic so wouldn't require much cpu, but I thought application load balancers would handle A LOT of traffic, at least enough to enable my services to need to auto grow as currently network dies before they do.
(would this require a network load balancer in front of multiple ALBs pointing to the same cluster? thats the only thing I can think of, which is fair enough, but i wouldn't have expected to hit this bottleneck with so little traffic )
