How to create a bash script for autoscaling EC2 instances given the work volume of a SQS?

user_15411076 0
Viewed 119

I created a bash script with aws-cli that sends 1000 messages using SQS, now I want to create another one that runs in parallel and creates and destroys EC2 instances given this condition: Checks every 15 seconds: if (((ApproximateNumberOfMessages + 9)/10) - N running instances) > 0 creates an instance, else destroys an instance.

My first problem is that I don't know how to connect my SQS queue to a EC2 instance so it can process these messages. I tried following this tutorial: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-sending-messages-from-vpc.html, but I don't want to use a private VPC and security groups so I was wondering if there is a way to make it easier.

My questions are: Is it possible to do it just using a bash script instead of CloudWatch and Autoscaling Groups? How do I create a EC2 instance that is ready to process these messages?

aws-cli amazon-web-services amazon-ec2 bash
1 Answers

When you create an EC2 instance, it automatically gets an Elastic Network Interface (ENI, a virtual network card) for which AWS automatically assigns either a default security group, either another user created security group. You can not detach the default ENI, also you cannot have an ENI without a security group. Moreover, EC2 instances have to run inside a VPC, which can be private of public. Nevertheless, if you work with EC2 instances, you have to deal with security groups as well.

Is it possible to do it just using a bash script instead of CloudWatch and Autoscaling Groups?

It might be possible, but you will find yourself reinventing the wheel. Autoscaling does more than just adding/removing instances based on some condition. For example, it also makes sure that your instances are replaced if they become unhealthy or if they are terminated for some reason. For more info see AWS ASG FAQ.

How do I create a EC2 instance that is ready to process these messages?

You can't just start an instance and expect to process your messages. You need to have some code or some kind software deployed to it and configured to poll messages from your queues.

user_76611190
Related