Define specific docker-compose file to use for AWS Elastic Beanstalk Deployment

user_1577947 0
Viewed 3803

Before I run eb create command, how can I tell Elastic Beanstalk to use a DIFFERENT docker-compose file?

For example, my project directory:

HelloWorldDocker
├──.elasticbeanstalk
│  └──config.yml
├──app/
├──proxy/
└──docker-compose.prod.yml
└──docker-compose.yml
  1. My docker-compose.yml is what I use for local development
  2. My docker-compose.prod.yml is what I want to use for production

Is there a way to define this configuration before running the eb create command from the EB CLI?

Stating the obvious: I realize I could use docker-compose.yml for my production file and a docker-compose.dev.yml for my local development but then running the docker-compose up command becomes more tedious locally (ie: docker-compose -f docker-compose.dev.yml up --build...). Further, I'm mainly interested if this is even possible as I'm learning Elastic Beanstalk, and how I could do it if I wanted to.

EDIT / UPDATE: June 11, 2021

I attempted to rename docker-compose.prod.yml to docker-compose.yml in .ebextensions/docker-settings.config with this:

container_commands:
  rename_docker_compose:
    command: mv docker-compose.prod.yml docker-compose.yml

>eb deploy:

2021-06-11 16:44:45    ERROR   Instance deployment failed.
  For details, see 'eb-engine.log'.
2021-06-11 16:44:45    ERROR   Instance deployment: Both 
  'Dockerfile' and 'Dockerrun.aws.json' are missing in your
  source bundle. Include at least one of them. The deployment
  failed.

In eb-engine.log, I see:

2021/06/11 16:44:45.818876 [ERROR] An error occurred during 
  execution of command [app-deploy] - [Docker Specific Build
  Application]. Stop running the command. Error: Dockerfile and
  Dockerrun.aws.json are both missing, abort deployment

Based on my testing, this is due to AWS needing to call /bin/sh -c docker-compose config before getting to the later steps of container_commands.

Edit / Update #2

If I use commands instead of container_commands:

commands:
  rename_docker_compose:
    command: mv docker-compose.prod.yml docker-compose.yml
    cwd: /var/app/staging

it does seem to do the replacement successfully:

2021-06-11 21:40:44,809 P1957 [INFO] Command find_docker_compose_file
2021-06-11 21:40:45,086 P1957 [INFO] -----------------------Command Output-----------------------
2021-06-11 21:40:45,086 P1957 [INFO]    ./var/app/staging/docker-compose.prod.yml
2021-06-11 21:40:45,086 P1957 [INFO] ------------------------------------------------------------
2021-06-11 21:40:45,086 P1957 [INFO] Completed successfully.

but I still am hit with:

2021/06/11 21:40:45.192780 [ERROR] An error occurred during
  execution of command [app-deploy] - [Docker Specific Build 
  Application]. Stop running the command. Error: Dockerfile and 
  Dockerrun.aws.json are both missing, abort deployment

EDIT / UPDATE: June 12, 2021

I'm on a Windows 10 machine. Before running eb deploy command locally, I opened up Git Bash which uses MINGW64 terminal. I cdd to the prebuild directory where build.sh exists. I ran:

chmod +x build.sh

If I do ls -l, it returns:

-rwxr-xr-x 1 Jarad 197121 58 Jun 12 12:31 build.sh*

I think this means the file is executable.

I then committed to git.

I then ran eb deploy.

I am seeing a build.sh: permission denied error in eb-engine.log. Below is an excerpt of the relevant portion.

...
2021/06/12 19:41:38.108528 [INFO] application/zip

2021/06/12 19:41:38.108541 [INFO] app source bundle is zip file ...
2021/06/12 19:41:38.108547 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2021/06/12 19:41:38.108556 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /var/app/staging/
2021/06/12 19:41:38.149125 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/ successfully
2021/06/12 19:41:38.149142 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2021/06/12 19:41:38.149190 [INFO] Executing platform hooks in .platform/hooks/prebuild/
2021/06/12 19:41:38.149249 [INFO] Following platform hooks will be executed in order: [build.sh]
2021/06/12 19:41:38.149255 [INFO] Running platform hook: .platform/hooks/prebuild/build.sh
2021/06/12 19:41:38.149457 [ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPreBuildHooks]. Stop running the command. Error: Command .platform/hooks/prebuild/build.sh failed with error fork/exec .platform/hooks/prebuild/build.sh: permission denied 

2021/06/12 19:41:38.149464 [INFO] Executing cleanup logic
2021/06/12 19:41:38.149572 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1623526898,"severity":"ERROR"}]}]}

2021/06/12 19:41:38.149706 [INFO] Platform Engine finished execution on command: app-deploy
...

Any idea why I am getting a permission denied error?

My Conclusion From This Madness

Elastic Beanstalk's EB CLI eb deploy command does not zip files (the app_source_bundle it creates) correctly on Windows machines.

Proof

I was able to recreate Marcin's example by zipping it locally and manually uploading it through the Elastic Beanstalk online interface. When I do that and check the source bundle, it shows that build.sh does have executable permissions (-rwxr-xr-x).

[root@ip-172-31-11-170 deployment]# zipinfo app_source_bundle
Archive:  app_source_bundle
Zip file size: 993 bytes, number of entries: 5
drwxr-xr-x  3.0 unx        0 bx stor 21-Jun-13 03:08 .platform/
drwxr-xr-x  3.0 unx        0 bx stor 21-Jun-13 03:08 .platform/hooks/
drwxr-xr-x  3.0 unx        0 bx stor 21-Jun-13 03:08 .platform/hooks/prebuild/
-rwxr-xr-x  3.0 unx       58 tx defN 21-Jun-13 03:09 .platform/hooks/prebuild/build.sh
-rw-r--r--  3.0 unx       98 tx defN 21-Jun-13 03:08 docker-compose.prod.yml

When I initialize and create using the EB CLI and the exact same files, build.sh does NOT have executable permissions (-rw-rw-rw-).

[ec2-user@ip-172-31-5-39 deployment]$ zipinfo app_source_bundle
Archive:  app_source_bundle
Zip file size: 1092 bytes, number of entries: 5
drwxrwxrwx  2.0 fat        0 b- stor 21-Jun-12 20:32 ./
-rw-rw-rw-  2.0 fat       98 b- defN 21-Jun-12 20:08 docker-compose.prod.yml
-rw-rw-rw-  2.0 fat      993 b- defN 21-Jun-12 20:15 myzip.zip
drwxrwxrwx  2.0 fat        0 b- stor 21-Jun-12 20:08 .platform/hooks/prebuild/
-rw-rw-rw-  2.0 fat       58 b- defN 21-Jun-12 20:09 .platform/hooks/prebuild/build.sh

Therefore, I think this is a bug with AWS EB CLI deploy command in regards to how it zips files for Windows users.

amazon-elastic-beanstalk docker-compose amazon-web-services
2 Answers

You can't do this from command level. But I guess you could write container_commands script to rename your docker-compose file from docker-compose.dev.yml to docker-compose.yml:

You can use the container_commands key to execute commands that affect your application source code. Container commands run after the application and web server have been set up and the application version archive has been extracted, but before the application version is deployed.

UPDATE 12 Jun 2021

I tried to replicate the issue using simplified setup with just docker-compose.prod.yml and Docker running on 64bit Amazon Linux 2 3.4.1 EB platform.

docker-compose.prod.yml

version: "3"

services:
    client:
        image: nginx
        ports:
            - 80:80

I can confirm and reproduce the issue with container_commands. So in my tests, the solution was to setup prebuild deployment hook.

So my deployment zip had the structure:

├── docker-compose.prod.yml
└── .platform
    └── hooks
        └── prebuild
            └── build.sh

where

build.sh

#!/bin/bash

mv docker-compose.prod.yml docker-compose.yml

I also made the build.sh executable before creating deployment zip.

app_source_bundle permissions (zipinfo -l)

Zip file size: 1008 bytes, number of entries: 5
drwxr-xr-x  3.0 unx        0 bx        0 stor 21-Jun-12 07:37 .platform/
drwxr-xr-x  3.0 unx        0 bx        0 stor 21-Jun-12 07:37 .platform/hooks/
drwxr-xr-x  3.0 unx        0 bx        0 stor 21-Jun-12 07:38 .platform/hooks/prebuild/
-rwxr-xr-x  3.0 unx       77 tx       64 defN 21-Jun-12 07:24 .platform/hooks/prebuild/build.sh
-rw-r--r--  3.0 unx       92 tx       68 defN 21-Jun-12 07:01 docker-compose.prod.ym
user_2488230

I was able to circumvent this annoying bug by:

  1. Using git and AWS CodeCommit
  2. Running git add --chmod=+x .platform/hooks/prebuild/build.sh

This circumvents the Windows-related issue because:

When you configure CodeCommit with your EB CLI repository, the EB CLI uses the contents of the repository to create source bundles. When you run eb deploy or eb create, the EB CLI pushes new commits and uses the HEAD revision of your branch to create the archive that it deploys to the EC2 instances in your environment.

Source: Deploying from your CodeCommit repository

user_15779470
Related