AWS ECS - How to restart/kill task if healthcheck fails?

user_5556466 0
Viewed 592

I have a task in AWS ECS which is running a service (Airflow 2). I would like to set up a healthcheck as they recommend in docs to avoid issues if scheduler goes down.

In my docker-compose I have:

scheduler:
    image: my-custom-airflow2
    command: scheduler
    healthcheck:
      test: [ "CMD-SHELL", 'airflow jobs check --job-type SchedulerJob --hostname "$${HOSTNAME}"' ]
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    container_name: my-scheduler
    build:
      context: .
      dockerfile: Dockerfile
    env_file:
      - ./airflow.env
    volumes:
      - ./database_utils:/database_utils
      - ./dags:/opt/airflow/dags
      - ./logs:/opt/airflow/logs
    depends_on:
      - webserver

In my task definition I have this:

healthCheck: [
        {
          retries : 5
          command : [
            "CMD-SHELL",
            "airflow jobs check --job-type SchedulerJob --hostname \"$${HOSTNAME}\""
          ]
        },
      ],

How can I replicate the restart command in the docker-compose in the task definition? The goals is if the check is not succesfull kill the service and restart a new one.

amazon-ecs docker
1 Answers

The AWS ECS healthcheck docs says this:

If a task is run manually, and not as part of a service, the task will continue its lifecycle regardless of its health status. For tasks that are part of a service, if the task reports as unhealthy then the task will be stopped and the service scheduler will replace it.

So here the issue is to ensure task is running in a service. Otherwise is not restarted.

user_55564660
Related