How should/can I avoid duplicated parts of Docker compose file of multiple interdependent projects

user_4487542 0
Viewed 2747

Let's take for example a mobile application who depends on two or more API.

Each of these projects are separated in independent Git repositories. Then we have 3 repositories allowing us to develop each in parallel.

Each projects have their own dependencies:

  • First API, for example, requires a SQL database
  • Second API, requires a NoSQL database
  • The mobile app requires these two APIs

Now I want to "dockerize" all of these projects to simplify development environment and unify it between developers and/or production environment.

Currently in each project we can create a custom docker-compose.yml file working with each projects requirements.

For example in the 1st API

version: "3.7"

services:
  first_api:
    image: golang:1.13
    working_dir:
      - /src
    depends_on:
      - mysql
    volumes:
      - ".:/src"
    command: go run main.go

  mysql:
    image: mysql:5.7
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_USER_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE_NAME}

  adminer:
    image: adminer
    restart: always

The second API will have a similar docker-compose.yml file but with NoSQL DB instead.

Then in the mobile app repository we will have a docker-compose.yml file with a lot of duplicated code (and exactly same containers) because of its interdependence with the two other API, an some other file identicals (e.g .env files, entrypoint scripts if needed...).

The databases setup/seeding will be also done on 2 repositories too, that can be a little annoying.

The docker-compose.yml file will look like something like this:

version: "3.7"

services:
  app:
    build:
      context: .
      args:
        - IP=${IP}
    ports:
      - 19000:19000
      - 19001:19001
      - 19002:19002
    volumes:
      - ".:/app"
    depends_on:
      - first-api
      - second-api

  first-api:
    image: my-registry:5000/first-api
    ports:
      - 9009:3000
    depends_on:
      - mysql
    volumes:
      - ".env:/dist/.env"

  mysql:
    image: mysql:5.7
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_USER_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE_NAME}

  adminer:
    image: adminer
    restart: always
    ports:
      - 9099:8080

  second-api:
    image: my-registry:5000/second-api
    ports:
      - 9010:3000
    depends_on:
      - mongo

  mongo:
    image: mongo
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${MONGO_ROOT_USERNAME}
      MONGO_INITDB_ROOT_PASSWORD: ${MONGO_ROOT_PASSWORD}

  mongo-express:
    image: mongo-express
    restart: always
    ports:
      - 8081:8081
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: ${MONGO_ROOT_USERNAME}
      ME_CONFIG_MONGODB_ADMINPASSWORD: ${MONGO_ROOT_PASSWORD}

In fact in this final docker-compose file we have 4 containers definition totally identical with them inside API's configurations, we also have some environment variables duplicated and versioned in 2 repositories minimum.

Sometime we can also have Dockerfile duplicated, according to specifics cases, DB setup or something too.

Did I miss something somewhere in this Docker development environment setup that would allow me to avoid some duplication?

Is there a best practice or a recommendation to avoid this?

How companies with big interdependent micro-services architecture manage these interdependence?

docker-compose docker development-environment
2 Answers
user_1936370

I'd probably set this up by independently running your two individual services' Docker Compose files, and then running a proxy in front that ties them together. You can "borrow" the networks from other docker-compose.yml files. You more or less have this now; the proxy would be your "app" container, and you can use an external: reference to the other applications' default networks.

version: '3'
services:
  app:
    build: .
    environment:
      - IP=${IP}
    ports:
      - 19000:19000
      - 19001:19001
      - 19002:19002
    networks:
      - firstapi_default
      - secondapi_default
networks:
  firstapi_default:
    external: true
  secondapi_default:
    external: true

This approach also works if you have multiple services that each independently need a MySQL database backend; running a separate docker-compose up in each project directory will instantiate a new separate database for each. (In a microservice architecture you typically don't share data stores between services; they communicate only via their APIs.)

You'll pretty quickly hit scaling issue doing this if one of your backend services needs to call another, and Docker Compose might not be the right tool for this. Kubernetes is a significant commitment, but it would let you deploy each of these individual services into a separate namespace, and then use DNS names like first_api.first_namespace.svc.cluster.local to communicate between them.

user_100081730
Related