How can I update and ECS service adding an addition load balance to which a service talks to without downtime?

user_9950 0
Viewed 648

We use terraform to manage our AWS resources and have the code to change the service from one to two load balancers.

However, terraform wants to destroy the service prior to recreating it. AWS cli docs indicate the reason - the API can only modify LBs during service creation and not on update.

It seems we need a blue/green deploy with the one LB and two LB services existing at the same time on the same cluster. I expect we'll need to create multiple task sets and the rest of the blue/green approach prior to this change (we'd planned for this anyway just not at this moment)

Does anyone have a good example for this scenario or know of any other approaches beyond full blue/green deployment?

blue-green-deployment terraform-provider-aws amazon-ecs
2 Answers

Alas, it is not possible to change the number of LBs during an update. The service must be destroyed and recreated.

Ideally, one would be doing blue green deploys with multiple ECS clusters and a set of LBs. Then cluster A could have the old service and cluster B have the new service allowing traffic to move from A to B as we go from blue to green.

We aren't quite there yet but plan to be soon. So, for now, we will do the classic parking lot switch approach:

In this example the service that needs to go from 1 LB to 2 LBs is called target_service

  1. clone target_service to become target_service2
  2. deploy microservices that can talk to either target_service or target_service2
  3. verify that target_service and target_service2 are both handling incoming data and requests
  4. modify target_service infra-as-code to go from 1 to 2 LBs
  5. deploy modified target_service (terraform deployment tool will destroy target_service leaving target_service2 to cover the gap, and then it will deploy target_service with 2 LBs
  6. verify that target_service with 2 LBS works and handle requests
  7. destroy and remove target_service2 as it is no longer needed

So, this is a blue-green like deploy albeit less elegant.

user_99500

Update 2022

ECS now supports updation of load balancers of a service. For now, this can be achieved using AWS CLI, AWS SDK, or AWS Cloudformation (not from AWS Console). From the documentation:

When you add, update, or remove a load balancer configuration, Amazon ECS starts new tasks with the updated Elastic Load Balancing configuration, and then stops the old tasks when the new tasks are running.

So, you don't need to create a new service. AWS update regarding this here. Please refer Update Service API doc on how to make the request.

user_19737350
Related