How to auto scale sudden request spike in AWS and GCP in cost effective way?

user_2667601 0
Viewed 189

We have SaaS application and we have thousands of customers. When our customers website get traffic then we also get same traffic as we are tracking activities of our customer's website visitors.

We couldn't get at which time we get sudden spike and all of our servers got down when we get sudden request spike due to traffic in our customer's website. To handle this we have configured to scale when our CPU or memory usage go beyond 60%. Which means we are paying 40% extra cost for unused resource. If we set it as 90% then our all servers became unresponsive due to sudden load and resource usage.

Instead of scale at 60%, we want to utilise at least 90% of resource we are paying for. Is there any better way to do scaling in cost effective way?

Note : We are using AWS ElasticBeanstalk and also GoogleCloud's Kubernetes Engine services.

gcp-load-balancer autoscaling aws-auto-scaling aws-load-balancer amazon-web-services
1 Answers

Instead of scale at 60%, we want to utilise at least 90% of resource we are paying for

90% is a pretty steep requirement. Then you need to set the scaling threshold to the 90% level. Apparently you see the problem if you will be unable to scale quick enough or fail to estimate the anticipated load. 60% sounds safe approach (and more expensive). Still - what's wrong with playing with the threshold?

To fully align with the load you may go serverless (both AWS and GCP offers sort of serverless functions). Under the high permanent load they may be more expensive, but if your problem is ever-changing load and overprovisioning, then the functions are ingenious answer.

Another approach is using the asynchronous processing, such as queues or streams and process the data at your own pace with resources you have. You may introduce some latency between receiving data and producing the results, but it may be acceptable in most of the cases.

user_16457120
Related