How to run docker-compose on AWS CodeBuild?

user_18706 0
Viewed 18208

I'm trying to setup automated Rails tests on AWS CodeBuild using docker-compose, but it errors out.

In buildspec.yml:

phases:
  build:
    commands:
        - docker-compose up -d

[Container] 2018/10/23 11:27:56 Running command docker-compose up -d
Couldn't connect to Docker daemon at http+docker://localhost - is it running?
 If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
 [Container] 2018/10/23 11:27:56 Command did not exit successfully docker-compose up -d exit status 1
[Container] 2018/10/23 11:27:56 Running command echo This always runs even if the install command fails
This always runs even if the install command fails
 [Container] 2018/10/23 11:27:56 Phase complete: BUILD Success: false
[Container] 2018/10/23 11:27:56 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker-compose up -d. Reason: exit status 1

Presumably I need to install docker and start the service, but that would be running Docker inside Docker and would require the build server to be started with privileged permission. Only examples I can see are for building Docker images, but I'm just trying to use it to setup the environment to run the test in.

ANSWERED: Set up Docker image in CodeBuild's Environment section

Thanks to @mferre for answering this. Docker-compose is indeed completely supported without doing anything special. The key is to choose a Docker image in the "environment" section when setting up inside AWS CodeBuild console (or same via the API):

enter image description here

Or can also be specified for an existing project - from Build / Build Projects, select the project, and Environments from the Edit menu. This lets you specify the image:

enter image description here

You could use any other image and script the Docker setup in buildspec.yml, but the easiest way is to use the official Docker image as above. With this as the container, docker and docker-compose are pre-installed, so docker-compose "just works". If the project has a docker-compose.yml file in its root, the buildspec.yml can be as simple as running it immediately:

version: 0.2
phases:
  build:
    commands:
      - docker-compose up -d
aws-codebuild docker-compose docker amazon-web-services ruby-on-rails
2 Answers

Okay, I figured out the issue!

You need to enable 'Privileged Access' on the CodeBuild container. This will allow you to interact with the docker cli.

Then add these two lines to the install commands:

- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2& - timeout 15 sh -c "until docker info; do echo .; sleep 1; done" `

ex:

version: 0.2

phases:
  install:
    commands:
      - nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2&
      - timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
  pre_build:
    commands:
      - docker build -t helloworld .
  build:
    commands:
      - docker images
      - docker run helloworld echo "Hello, World!"
user_2012970

Are you using the Docker runtime for your build environment?

I am working with this repo: https://github.com/mreferre/yelb

That dockercompose isn't building anything (it's rather meant to deploy) but I believe you are hitting a problem much before that. Your compose up should work if you use the Docker runtime.

This is what my buildspec looks like: 

version: 0.2
phases:
  build:
    commands:
        - cd deployments/platformdeployment/Docker
        - docker-compose up -d

And this is the output: 

[Container] 2019/02/20 13:48:02 Waiting for agent ping 
[Container] 2019/02/20 13:48:04 Waiting for DOWNLOAD_SOURCE 
[Container] 2019/02/20 13:48:07 Phase is DOWNLOAD_SOURCE 
[Container] 2019/02/20 13:48:07 CODEBUILD_SRC_DIR=/codebuild/output/src292484508/src/github.com/mreferre/yelb 
[Container] 2019/02/20 13:48:07 YAML location is /codebuild/readonly/buildspec.yml 
[Container] 2019/02/20 13:48:07 Processing environment variables 
[Container] 2019/02/20 13:48:07 Moving to directory /codebuild/output/src292484508/src/github.com/mreferre/yelb 
[Container] 2019/02/20 13:48:07 Registering with agent 
[Container] 2019/02/20 13:48:07 Phases found in YAML: 1 
[Container] 2019/02/20 13:48:07  BUILD: 2 commands 
[Container] 2019/02/20 13:48:07 Phase complete: DOWNLOAD_SOURCE Success: true 
[Container] 2019/02/20 13:48:07 Phase context status code:  Message:  
[Container] 2019/02/20 13:48:07 Entering phase INSTALL 
[Container] 2019/02/20 13:48:07 Phase complete: INSTALL Success: true 
[Container] 2019/02/20 13:48:07 Phase context status code:  Message:  
[Container] 2019/02/20 13:48:08 Entering phase PRE_BUILD 
[Container] 2019/02/20 13:48:08 Phase complete: PRE_BUILD Success: true 
[Container] 2019/02/20 13:48:08 Phase context status code:  Message:  
[Container] 2019/02/20 13:48:08 Entering phase BUILD 
[Container] 2019/02/20 13:48:08 Running command cd deployments/platformdeployment/Docker 

[Container] 2019/02/20 13:48:08 Running command docker-compose up -d 
Creating network "docker_yelb-network" with driver "bridge" 
Pulling redis-server (redis:4.0.2)... 
4.0.2: Pulling from library/redis 
Pulling yelb-db (mreferre/yelb-db:0.3)... 
0.3: Pulling from mreferre/yelb-db 
Pulling yelb-appserver (mreferre/yelb-appserver:0.3)... 
0.3: Pulling from mreferre/yelb-appserver 
Pulling yelb-ui (mreferre/yelb-ui:0.3)... 
0.3: Pulling from mreferre/yelb-ui 
Creating docker_redis-server_1 ...  
Creating docker_yelb-db_1      ...  
·[2A·[2K 
Creating docker_redis-server_1 ... ·[32mdone·[0m 
·[2B·[1A·[2K 
Creating docker_yelb-db_1      ... ·[32mdone·[0m 
·[1BCreating docker_yelb-appserver_1 ...  
·[1A·[2K 
Creating docker_yelb-appserver_1 ... ·[32mdone·[0m 
·[1BCreating docker_yelb-ui_1        ...  
·[1A·[2K 
Creating docker_yelb-ui_1        ... ·[32mdone·[0m 
·[1B 
[Container] 2019/02/20 13:49:00 Phase complete: BUILD Success: true 
[Container] 2019/02/20 13:49:00 Phase context status code:  Message:  
[Container] 2019/02/20 13:49:00 Entering phase POST_BUILD 
[Container] 2019/02/20 13:49:00 Phase complete: POST_BUILD Success: true 
[Container] 2019/02/20 13:49:00 Phase context status code:  Message:
user_31467410
Related