Unable to run a health check for mysql image on AWS ECS task

user_4955976 0
Viewed 220

I have a cluster on AWS and it has a task that should run two containers. One of the containers called app has a dependency on my db container which is a mysql image and it should run when my db image is healthy.

I have managed to write a healthcheck in a docker compose with command

["CMD","mysqladmin","ping","-h","localhost"].

However I'm unable to run the same command on my aws container and the container is always unhealthy when I write this command on my task definition.

I have replaced CMD with CMD-Shell and it has the same result. I have also tried the generic healthcheck command advised by AWS [ "CMD-SHELL", "curl -f http://localhost/ || exit 1" ] but since the mysql image does not have curl installed I get the same result again.

What am I doing wrong?

amazon-ecs health-check amazon-web-services containers mysql
1 Answers

I'm using exactly the same command (CMD) in my test deployment and it works, this makes me think that maybe you're interpreting the unhealthy state as the health check command not working, but there are a number of other reasons for that state.

For example, during my tests it wasn't able to start because I was using EFS to provide permanent storage and every time I was updating the task, the files were not deleted, so the container was failing because it was Unable to lock ./ibdata1.

There's apparently a second issue anyway, because CMD and CMD-SHELL (which are part of the docker syntax and unrelated the OS), requires a different syntax: CMD need a list, while CMD-SHELL allows a single string as the second item of the array.

Basically, if you try this in compose, it works:

["CMD","mysqladmin","ping","-h","localhost"]

this does not:

["CMD-SHELL","mysqladmin","ping","-h","localhost"]

and it should be instead:

["CMD-SHELL","mysqladmin ping -h localhost"]

So to answer the question, make sure to create a log group on Cloudwatch and set the "logConfiguration" into the container definition and check the logs, more than likely there's something wrong into that container, and make sure to use the right syntax into the container definition, either this example

      "healthCheck": {
          "retries": 3,
          "command": ["CMD","mysqladmin","ping","-h","localhost"],
          "timeout": 5,
          "interval": 5,
          "startPeriod": null
      },

or this one - provided a shell has been defined into the container

      "healthCheck": {
          "retries": 3,
          "command": ["CMD-SHELL","mysqladmin ping -h localhost"],
          "timeout": 5,
          "interval": 5,
          "startPeriod": null
      },

Into your logs you should find a lot of record like

[Note] Access denied for user 'root'@'localhost' (using password: NO)

Which is the healthcheck running. It fails because we didn't set any password, but as the mysqladmin docs are stating:

The return status from mysqladmin is 0 if the server is running, 1 if it is not. This is 0 even in case of an error such as Access denied, because this means that the server is running but refused the connection, which is different from the server not running.

user_43176880
Related