How to avoid race condition during docker overlay network creation?

This issue is still not resolved, but I was easily able to avoid it using the run-one command (instead of run command, it becames run-one run command, and return an error if the command is still running).

(You can verify if the run-one command is available with which run-one)

Steps:

1. Create a script that creates the network (it may accept the network name as a parameter, like docker network create "$1").
2. Create the network (wherever it should be created) by calling the script with run-one to make sure that it is not executing twice for the same network (run /path/to/script network-name).
3. ?
4. Profit!

You can see this approach in action in the (demo) script below:

#!/bin/bash
set -eou pipefail

RED='\033[0;31m'
NC='\033[0m' # No Color

function error {
    msg="$(date '+%F %T') - ${BASH_SOURCE[0]}:${BASH_LINENO[0]}: ${*}"
    >&2 echo -e "${RED}${msg}${NC}"
    exit 2
}

file="${BASH_SOURCE[0]}"

command="${1:-}"

if [ -z "$command" ]; then
    error "[error] no command entered"
fi

shift;

case "$command" in
    "clean")
        sudo docker network prune -f
        ;;
    "test1")
        run-one "$file" "test:concurrent" "test:network"
        ;;
    "test2")
        run-one "$file" "test:concurrent" "test:network:unique"
        ;;
    "test:concurrent")
        echo "===========before==========="
        sudo docker network ls
        echo "============================"

        cmd="$1"

        pids=()

        for i in $(seq 1 3); do
            "$file" "$cmd" &
            pids["${i}"]=$!
        done

        idx=0

        for pid in "${pids[@]}"; do
            wait "$pid" && status="$?" || status="$?"
            idx=$((idx + 1))

            if [ "$status" != '0' ]; then
                echo "error in process $pid (#$idx)"
            fi
        done

        echo "===========after============"
        sudo docker network ls
        echo "============================"
        ;;
    "test:network:unique")
        run-one "$file" "test:network"
        ;;
    "test:network")
        sudo docker network create "my-network"
        ;;
    *)
        echo -e "${RED}[error] invalid command: $command${NC}"
        exit 1
        ;;
esac

Then:

1. Run /path/to/script clean to remove unused networks (make sure to run this script in a development environment).
2. Run /path/to/script test1 and see that there are 3 networks named my-network.
3. Run /path/to/script clean again.
4. Run /path/to/script test2 and see that there is only 1 network named my-network (2 of the 3 processes ended up with an error due to the run-one command, with only one creating the network).

The fact that the script adds another abstraction layer (that may add complexity if you intend to use network options), aside from the fact that you must create the script and refer to it, makes this solution to be described at best as a workaround.

That said, this is easily achievable and I don't think this should be labeled as a hack, although a proper solution IMO should be on the docker engine side (probably in the API).

This may not be so easily achievable with docker-compose tough, unless you run it from a script that you can easily change and you know the names of the networks beforehand.