I've two EC2 instances behind an ELB and in an Auto Scaling Group. The Scale-up policy is as below:
CPUUtilization >= 70 for 300 seconds (Adds one server)
While Atoscaling activity is taking place, load on existing instances is already 99% and connections are being dropped.
Is there any way to handle this more efficiently?
Check the status of your instances whether they are healthy or not. Usually instances / container instances go into unhealthy / draining static if there is sudden overwhelming traffic.
In that case, set up an autoscaling policy to detect the minimum number healthy instances being dropped and accordingly scale your instances to atleast 50% more of your minimum threshold. For example, a system running with 30 healthy instances suddenly drops to 25, then autoscale to additional 45 instances so that it can handle that sudden spike.
Later as the traffic cools down, your scale down policy on metrics like CPU, Memory can bring those 45 instances down to required number.
