DNS resolution failing inside docker container when host network changes without rebooting device

Viewed 1094

Expected behavior

DNS resolution failing inside docker container when host network changes ( network1-192.168.1., network2-192.168.2. ) without rebooting device docker container running on bridge network

Actual behavior

DNS resolution should be successful in case of host network changes

Steps to reproduce the behavior

  1. Create network container (lets say.. "abc_network.ctr") without specifying network_mode as it takes bridge by default (and it used docker's embedded nameserver inside docker container)
  2. Create another container (xyz.ctr) whose network mode will be "container" ("abc_network.ctr")
  3. /etc/resolv.conf from inside the xyz.ctr container. which is able to resolve nameserver
    root@d3e0dc71f370:~# cat /etc/resolv.conf 
    nameserver 127.0.0.11
    options ndots:0

  1. /etc/resolv.conf from host.. which is also able to resolve nameserver
    root@RP4-2711-c03112-1xb0e6f3b8:~# cat /etc/resolv.conf
    # This file is managed by man:systemd-resolved(8). Do not edit.
    #
    # This is a dynamic resolv.conf file for connecting local clients directly to
    # all known uplink DNS servers. This file lists all configured search domains.
    #
    # Third party programs must not access this file directly, but only through the
    # symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
    # replace this symlink by a static file or a different symlink.
    #
    # See man:systemd-resolved.service(8) for details about the supported modes of
    # operation for /etc/resolv.conf.
    
    nameserver 192.168.1.1

  1. Change network subnet of the machine (remove LAN cable and connect to another subnet network1 -> network2) without reboot.

  2. And try to resolve nameserver inside docker container using following commands:

  • docker exec -it xyz.ctr bash
  • apt-get update result-
    root@d3e0dc71f370:~# apt-get update
    Err:1 http://security.debian.org/debian-security buster/updates InRelease
      Temporary failure resolving 'security.debian.org'
    Err:2 http://deb.debian.org/debian buster InRelease                  
      Temporary failure resolving 'deb.debian.org'
    Err:3 http://deb.debian.org/debian buster-updates InRelease
      Temporary failure resolving 'deb.debian.org'
    Reading package lists... Done    
    W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease  Temporary failure resolving 'deb.debian.org'
    W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease  Temporary failure resolving 'security.debian.org'
    W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease  Temporary failure resolving 'deb.debian.org'
    W: Some index files failed to download. They have been ignored, or old ones used instead.

  • dig output

     root@15c396b06f57:~# dig
    
    ; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>>
    ;; global options: +cmd
    ;; connection timed out; no servers could be reached

  1. /etc/resolve.conf from host machine which is able to resolve nameserver

    # This file is managed by man:systemd-resolved(8). Do not edit.
    #
    # This is a dynamic resolv.conf file for connecting local clients directly to
    # all known uplink DNS servers. This file lists all configured search domains.
    #
    # Third party programs must not access this file directly, but only through the
    # symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
    # replace this symlink by a static file or a different symlink.
    #
    # See man:systemd-resolved.service(8) for details about the supported modes of
    # operation for /etc/resolv.conf.
    
    nameserver 192.168.2.1

  1. apt-get working on host and dig command shows NO ERROR

Output of docker version:


    Docker version 20.10.6, build 370c289

Output of docker info:

    root@RP4-2711-c03112-1xb0e6f3b8:~# docker info
    Client:
     Context:    default
     Debug Mode: false
     Plugins:
      app: Docker App (Docker Inc., v0.9.1-beta3)
      buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
    
    Server:
     Containers: 22
      Running: 18
      Paused: 0
      Stopped: 4
     Images: 98
     Server Version: 20.10.6
     Storage Driver: overlay2
      Backing Filesystem: extfs
      Supports d_type: true
      Native Overlay Diff: true
      userxattr: false
     Logging Driver: json-file
     Cgroup Driver: cgroupfs
     Cgroup Version: 1
     Plugins:
      Volume: local
      Network: bridge host ipvlan macvlan null overlay
      Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
     Swarm: inactive
     Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
     Default Runtime: runc
     Init Binary: docker-init
     containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
     runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
     init version: de40ad0
     Security Options:
      seccomp
       Profile: default
     Kernel Version: 5.10.17-v7l+
     Operating System: Raspbian GNU/Linux 10 (buster)
     OSType: linux
     Architecture: armv7l
     CPUs: 4
     Total Memory: 3.44GiB
     Name: RP4-2711-c03112-1xb0e6f3b8
     ID: C66C:3DAT:6DLH:PHJA:JR2N:KURB:PTMQ:BHUP:XRIH:FE3H:VSEZ:B7K3
     Docker Root Dir: /var/lib/docker
     Debug Mode: false
     Registry: https://index.docker.io/v1/
     Labels:
     Experimental: false
     Insecure Registries:
      127.0.0.0/8
     Live Restore Enabled: false
     Default Address Pools:
       Base: 172.128.0.1/10, Size: 16
    
    WARNING: No memory limit support
    WARNING: No swap limit support
    WARNING: No kernel memory TCP limit support
    WARNING: No oom kill disable support

Additional environment details (AWS, VirtualBox, physical, etc.)

Device- Raspberry Pi-4B, Intel NUC (This is not an environment specific issue)

0 Answers
Related