Docker scale with deterministic port binding

user_4823977 0
Viewed 1461

I would like to scale a wildfly container having exposed multiple ports with deterministic results.

docker-compose.yml

version: '3'
services:
  wildfly-server:
    build:
      context: .
      dockerfile: Dockerfile
      args:
        admin_user: admin
        admin_password: admin
    deploy:
      resources:
          limits:
            memory: 1.5G
            cpus: "1.5"
    restart: always
    ports:
      - "8000-8099:8080"
      - "8100-8199:9990"
      - "8200-8299:8787"
    expose:
      - "8080"
      - "9990"
      - "8787"

Dockerfile

FROM jboss/wildfly:16.0.0.Final

# DOCKER ENV VARIABLES
ENV WILDFLY_HOME /opt/jboss/wildfly
ENV STANDALONE_DIR ${WILDFLY_HOME}/standalone
ENV DEPLOYMENT_DIR ${STANDALONE_DIR}/deployments
ENV CONFIGURATION_DIR ${STANDALONE_DIR}/configuration

RUN ${WILDFLY_HOME}/bin/add-user.sh ${admin_user} ${admin_password} --silent

# OPENING DEBUG PORT
RUN rm ${WILDFLY_HOME}/bin/standalone.conf
ADD standalone.conf ${WILDFLY_HOME}/bin/

# SET JAVA ENV VARS
RUN rm ${CONFIGURATION_DIR}/standalone.xml
ADD standalone.xml ${CONFIGURATION_DIR}/

Command to start

docker-compose up --build --force-recreate --scale wildfly-server=10

It almost works as I want to, but there is some port discrepancy. When I create the containers, I want them to have incremental ports for each container to be exposed as follows:

machine_1 8001, 8101, 82001
machine_2 8002, 8102, 82002
machine_3 8003, 8103, 82003

But what I get as a result is not deterministic and looks like this:

machine_1 8001, 8102, 82003
machine_2 8002, 8101, 82001
machine_3 8003, 8103, 82002

The problem is that every time I run the compose up command, the ports are different for each container.

Example output:

CONTAINER ID  COMMAND                  CREATED             STATUS              PORTS                                                                    NAMES
0232f24fbca4  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8028->8080/tcp, 0.0.0.0:8231->8787/tcp, 0.0.0.0:8126->9990/tcp   wildfly-server_7
13a6a365a552  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8031->8080/tcp, 0.0.0.0:8230->8787/tcp, 0.0.0.0:8131->9990/tcp   wildfly-server_10
bf8260d9874d  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8029->8080/tcp, 0.0.0.0:8228->8787/tcp, 0.0.0.0:8129->9990/tcp   wildfly-server_6
3d58f2e9bdfe  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8030->8080/tcp, 0.0.0.0:8229->8787/tcp, 0.0.0.0:8130->9990/tcp   wildfly-server_9
7824a73a09f5  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8027->8080/tcp, 0.0.0.0:8227->8787/tcp, 0.0.0.0:8128->9990/tcp   wildfly-server_3
85425462259d  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8024->8080/tcp, 0.0.0.0:8224->8787/tcp, 0.0.0.0:8124->9990/tcp   wildfly-server_2
5be5bbe8e577  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8026->8080/tcp, 0.0.0.0:8226->8787/tcp, 0.0.0.0:8127->9990/tcp   wildfly-server_8
2512fc0643a3  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8023->8080/tcp, 0.0.0.0:8223->8787/tcp, 0.0.0.0:8123->9990/tcp   wildfly-server_5
b156de688dcb  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8025->8080/tcp, 0.0.0.0:8225->8787/tcp, 0.0.0.0:8125->9990/tcp   wildfly-server_4
3e9401552b0a  "/opt/jboss/wildfly/…"   5 minutes ago       Up 5 minutes        0.0.0.0:8022->8080/tcp, 0.0.0.0:8222->8787/tcp, 0.0.0.0:8122->9990/tcp   wildfly-server_1

Question

Is there any way to make the port distribution deterministic? Like disable parallel running to have serial checks on the available ports or any other method? The only alternative I found is to have a yml template and generate all the necessary files (like 10 if I need 10 containers etc). Are there any alternative solutions?

docker-compose docker
2 Answers

No, you cannot currently (10/14/19) make the port selection deterministic in the docker-compose file. This behavior was requested in Github issues #722 and #1247, but those issues were closed without the issue having been implemented.

If you want to semi-dynamically scale an application like it sounds like you do, then you'll need to solve this another way. Your .yml templating idea sounds like the cleanest solution IMO.

Are you sure you need the ports to be deterministic? If you use a reverse proxy like nginx that listens on one host port and balances the load between all of your docker containers, would that work for your use case? Setting up an nginx load balancer in a docker container is pretty straightforward. I suggest you look into that, and if you still need a deterministic way for a caller to know the service's port so it can send a request to a specific server repeatedly, then go with your .yml templating solution or some kind of service discovery process separate from the docker-compose configuration.

user_40886420

You could do this using variable substitution:

yaml:
...
   ports:
      ${PORT}:8080

Then call docker with the specific ports:

for p in {8000..8099}; do
   PORT=$p docker-compose ...
done
user_14807400
Related