Invalid IP address in add-host: "" + Failed to program FILTER chain: iptables failed

Viewed 17653

I am working with a scientific code which was packaged in Docker containers by another person. I am not very familiar with all the magic behind containers, images etc. And I was using it only by running some simple commands like docker-compose up or docker-compose up --build if I needed to add some dependencies in the code.
So everything was perfectly normal until last night. I was running a simulation that took a whole night, but I saw that results were not ok, so I just killed the processed by pressing ctrl + C 2 or 3 times. When I tried to launch simulation again by docker-compose up I got an error which, unfortunately, I can't recall right now. Also, strange thing - at that moment I couldn't connect to Internet. I rebooted, Internet worked fine again, I tried to run docker-compose up again and I got the following output:

WARNING: The DOCKERHOST variable is not set. Defaulting to a blank string.
Creating alcor_alcor_1 ... 
alcor_cassandra_1 is up-to-date
Creating alcor_alcor_1 ... error

ERROR: for alcor_alcor_1  Cannot create container for service alcor: invalid IP address in add-host: ""

ERROR: for alcor  Cannot create container for service alcor: invalid IP address in add-host: ""
ERROR: Encountered errors while bringing up the project.

I deleted all the images, containers and volumes by running:
docker rm $(docker ps -a -f status=exited -q)
docker rmi $(docker images -a -q)
docker volume rm $(docker volume ls -f dangling=true -q)

And then I rebuilt everything by running: docker-compose build --no-cache
No errors were thrown. I ran docker-compose up again and still got the same error: invalid IP address in add-host: "" Then I repeated my steps again: deleted all images, containers and volumes, rebuilt everything and tried to run again. Now every time it shows me this:

WARNING: The DOCKERHOST variable is not set. Defaulting to a blank string.
Creating network "alcor_default" with the default driver
ERROR: Failed to program FILTER chain: iptables failed: iptables -I FORWARD -o br-231cf5f5b939 -j DOCKER: iptables v1.4.14: Couldn't load target `DOCKER':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.
  (exit status 2)

What could be the reason for this and how do I resolve it? Can I provide any additional information to help understand this? Google and Docker docs don't say anything on this matter. And I didn't find anything on SO about it. Any help is appreciated.

Here is something that probably could help understand the issue:

$ docker version
Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:27:03 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:22:34 2017
 OS/Arch:      linux/amd64
 Experimental: false

docker-compose.yml

version: '3'

services:
  alcor:
    build: .
    image: lycantropos/alcor:latest
    entrypoint: "/alcor/docker-entrypoint.sh"
    volumes:
      - .:/alcor/
    extra_hosts:
      - "dockerhost:$DOCKERHOST"  # for debugging
    command:
      # Here I omitted many options and arguments
      - simulate
    environment:
      - CASSANDRA_RPC_ADDRESS=cassandra
      - CASSANDRA_RPC_PORT=9042

  cassandra:
    image: cassandra:latest
    volumes:
      - cassandra-data:/var/lib/cassandra

volumes:
  cassandra-data:

Edit: I tried again. I cleaned everything, rebooted and rebuilt, and the 1st error returned.

3 Answers

You may try this steps before disabling the firewall:

export DOCKERHOST=$(ifconfig | grep -E "([0-9]{1,3}\.){3}[0-9]{1,3}" | grep -v 127.0.0.1 | awk '{ print $2 }' | cut -f2 -d: | head -n1)

docker-compose down --remove-orphans

docker-compose build

docker-compose up

I did a bit similar to @noreddine-belhadj-cheikh, but with a slight different commands that might also work on Debian and other distros better. I also had a launch script (up.sh) that does the following:

export DOCKERHOST=$(ip addr show | grep "\binet\b.*\bdocker0\b" | awk '{print $2}' | cut -d '/' -f 1)
docker-compose up
Related