Swarm node Status down, but node should be Ready

Viewed 9282

I am trying to run a service on a swarm composed of three Raspberry PIs.
I have one manager and two worker nodes.

The problem is that sometimes the status of the worker nodes is "Down" even if the nodes are correctly switched on and connected to the network.

I just started using Docker so I might be doing something wrong, but everything seems to be correctly set.
How would you avoid that "Down" status?

4 Answers

I've had the same issue before. You can fix it by cleaning up /var/lib/docker/swarm/ on the problematic node, then reattach it to the swarm.

1) on problem node 

sudo systemctl stop docker
sudo rm -rf /var/lib/docker/swarm

2) on swarm manager 

docker node rm <problem-node-name>
docker swarm join-token worker
    docker swarm join --token <token> <manager_ip>:2377

3) on problem node 

sudo systemctl start docker
enter code here
docker swarm join --token <token> <manager_ip>:2377

 

In my case, the docker node had invalid default route and DNS did not work. I was anyways able to ssh on the machine by ip address. I tested first:

ping google.com

Which did not work. Then I changed the default route:

route -n
route add default gw 10.1.2.3
route del default gw 10.1.2.1 (offending gateway)    

And finally changed the DNS server from:

/etc/resolv.conf

Then the node came up automatically.

In my case, (virtual) network devices changed. Just adjusted settings, did docker swarm leave and docker swarm join for each of the nodes with the problem and then from the manager I removed (docker node rm ...) them. Worked without issues after that.

Related