docker run dynamodb-local on Github Actions Workflow hanging

Question

I'm currently working on a small CICD project that will run a series of tests on Github Actions using dynamodb-local whenever I update my code and then package and deploy if the tests are successful.

I have the following workflow:

name: backend_actions
on:
  workflow_dispatch:
  push:
    paths:
      - 'backend/*'
    branches:
      - master
jobs:
  test-locally:
    runs-on: ubuntu-latest
    outputs:
      test-result: ${{ steps.run-tests.outputs.result }}
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.9'
      - uses: aws-actions/setup-sam@v1
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-west-2
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - name: Setup local DynamoDB
        run: docker run -p 8000:8000 amazon/dynamodb-local
      - name: Create table
        run: aws dynamodb create-table --cli-input-json file://backend/src/test/make_table.json --endpoint-url http://localhost:8000
      - name: start local API Gateway
        run: sam local start-api --env-vars backend/env.json
      - id:   run-tests
        name: Run tests
        run: |
          python backend/src/test_dynamoDB_lambda.py
          echo "::set-output name=result::$?"
  update_backend:
    needs: test-locally
    if: ${{ needs.test-locally.outputs.test-result == '0' }}
    runs-on: ubuntu-latest
    steps:
      - name: Package and deploy
        run: |
          aws cloudformation package --s3-bucket cloud-resume-bucket \
          --template-file backend/template.yaml --output-template-file backend/gen/template-gen.yaml
          aws cloudformation deploy --template-file backend/gen/template-gen.yaml --stack-name cloud-formation-resume \
          --capabilities CAPABILITY_IAM

When I try running the workflow in Github Actions, it will get to the 'Setup local DynamoDB' step, output the text below, and then hang.

Run docker run -p 8000:8000 amazon/dynamodb-local
Unable to find image 'amazon/dynamodb-local:latest' locally
latest: Pulling from amazon/dynamodb-local
2cbe74538cb5: Pulling fs layer
137077f50205: Pulling fs layer
58932e640a40: Pulling fs layer
58932e640a40: Verifying Checksum
58932e640a40: Download complete
2cbe74538cb5: Verifying Checksum
2cbe74538cb5: Download complete
137077f50205: Verifying Checksum
137077f50205: Download complete
2cbe74538cb5: Pull complete
137077f50205: Pull complete
58932e640a40: Pull complete
Digest: sha256:bdd26570dc0e0ae49e1ea9d49ff662a6a1afe9121dd25793dc40d02802e7e806
Status: Downloaded newer image for amazon/dynamodb-local:latest
Initializing DynamoDB Local with the following configuration:
Port:   8000
InMemory:   true
DbPath: null
SharedDb:   false
shouldDelayTransientStatuses:   false
CorsParams: *

Seems like it can find the docker image and download it fine, but stops upon initializing? This is my first time working with Github Actions and Docker, so I'm not really sure why it's hanging on Github Actions and not when I run it on my own computer, so any help would be appreciated!

user_1208104 · Answer

When you run the command docker run -p 8000:8000 amazon/dynamodb-local the process never exits, so the Github run block doesn't actually know when to move on to the next step—it just hangs there forever.

What I did in my project is simply background it, by using the & after the command:

      - name: Setup local DynamoDB
        run: docker run -p 8000:8000 amazon/dynamodb-local &

Github Workflows will start the Docker container and move to the next run step, and when all the steps are done it'll just kill the container as part of normal cleanup. Because of this, you don't need to worry about shutting it down at the end.

The difficulty with this approach is that it takes several seconds for DynamoDB-local to start up, but your next step relies on it and will likely throw ECONNREFUSED errors.

What I've done in my project is to have the next run step execute a script that attempts to list tables, retrying with a short delay until it gets back a response.

The bash command is simply (you would need to put this in a while+try/catch loop):

aws dynamodb list-tables --endpoint-url http://localhost:8000

As a guide, this is (roughly) what I do in JavaScript, using the aws-sdk and NodeJS@16:

// wait-for-dynamodb.js
import timers from 'timers/promises'
import AWS from 'aws-sdk'

const dynamodb = new AWS.DynamoDB()

const waitForDynamoDbToStart = async () => {
    try {
        await dynamodb.listTables().promise()
    } catch (error) {
        console.log('Waiting for Docker container to start...')
        await timers.setTimeout(500)
        return waitForDynamoDbToStart()
    }
}

const start = Date.now()
waitForDynamoDbToStart()
    .then(() => {
        console.log(`DynamoDB-local started after ${Date.now() - start}ms.`)
        process.exit(0)
    })
    .catch(error => {
        console.log('Error starting DynamoDB-local!', error)
        process.exit(1)
    })

Then I simply have that in the run steps:

      - name: Setup local DynamoDB
        run: docker run -p 8000:8000 amazon/dynamodb-local &
      - name: Wait for it to boot up
        run: node ./wait-for-dynamodb.js
      # now you're guaranteed to have DynamoDB-local running

docker run dynamodb-local on Github Actions Workflow hanging

1 Answers