We have an issue remove ECS service target from target group and then remove the target group which is no longer needed via CDK , because the CDK will complain about that it is still in used by the ECS in ecs_service stack. We have our target group in one stack ecs_load_balancer and our ECS services in a different stack ecs_service. ecs_service stack depends on ecs_load_balancer.
When we break the dependency: props.targetGroup.addTarget(this.service); in ecs_service stack. CDK will try to delete the output of the target group and modify the target group resource (as I can see when I check the bb cdk diff). While CDK deploys ecs_load_balancer stack first, CDK will think that this resource target group is still in used and throw an error when it tries to delete/modifies the target group :
Export Personal-Dev-us-east-1-ACSCalling-EcsLoadBalancer:ExportsOutputRefLoadBalancerListenerTargetGroupGroup27D2B0EED93AD008 cannot be deleted as it is in use by Personal-Dev-us-east-1-ACSCalling-EcsService and Personal-Dev-us-east-1-ACSCalling-OnePod-EcsService
We can delete the ECS target from the target group on the AWS console. But not via CDK because of this cross stack reference problem.
I've tried to manually add back the Output that gets deleted in ecs_load_balancer. But I still can't get rid of this change which will still cause deployment failure since the it can't be modified:
[~] AWS::ElasticLoadBalancingV2::TargetGroup LoadBalancer/Listener/TargetGroupGroup LoadBalancerListenerTargetGroupGroup27D2B0EE replace
└─ [-] TargetType (requires replacement)
└─ ip
We couldn't find a way to get around this error from CDK. How can we remove this target group that we no longer need?