updates.jenkins.io: Temporary failure in name resolution while building jenkins docker image in docker in docker Jenkins agent running on K8S

user_1264304 0
Viewed 301

I'm using dynamically provisioned docker in docker agent as Kubernetes pod in Jenkins helm chart.

Docker in Docker agent uses 2 images, one with docker client, one with docker daemon. Their Dockerfile is below:

  • dind-client
FROM jenkins/jnlp-agent-docker
USER root

COPY entrypoint.sh /entrypoint.sh
RUN chown jenkins:jenkins /entrypoint.sh
RUN chmod +x /entrypoint.sh

USER jenkins
ENTRYPOINT "/entrypoint.sh"
  • entrypoint.sh busy waits for docker-daemon to become available:
#!/usr/bin/env bash

RETRIES=6

sleep_exp_backoff=1

for((i=0;i<RETRIES;i++)); do
    docker version
    dockerd_available=$?
    if [ $dockerd_available == 0 ]; then
        break 
    fi
    sleep ${sleep_exp_backoff}
    sleep_exp_backoff="$((sleep_exp_backoff * 2))"
done

exec /usr/local/bin/jenkins-agent "$@"
  • dind-daemon

FROM docker:20.10.14-dind

  • docker-agent is configured to run in Jenkins helm chart using below values in values.yaml:
dind:
    podName: docker-agent
    customJenkinsLabels: docker-agent
    image: [url]/docker-client
    tag: latest
    envVars:
     - name: DOCKER_HOST
       value: "tcp://localhost:2375"
    alwaysPullImage: true
    yamlTemplate:  |-  
     spec: 
         containers:
           - name: dind-daemon 
             image: [url]/docker-daemon:latest
             securityContext: 
               privileged: true
             env: 
               - name: DOCKER_TLS_CERTDIR
                 value: ""
  • I'm trying to build below image using the agent:
FROM docker.io/jenkins/jenkins:lts-jdk17

ENV JAVA_OPTS="-Djenkins.install.runSetupWizard=false"

# Add Jenkins init files
COPY src/ /usr/share/jenkins/ref/

COPY --chown=jenkins:jenkins plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli --verbose --plugin-file /usr/share/jenkins/ref/plugins.txt

USER jenkins
  • Below error pops during RUN jenkins-plugin-cli --verbose --plugin-file /usr/share/jenkins/ref/plugins.txt stage of docker build command inside the agent during jenkin build using the agent.
12:39:03  Retrieving update center information
12:39:03  Created cache at: /var/jenkins_home/.cache/jenkins-plugin-management-cli
12:39:03  Update center URL: https://updates.jenkins.io/update-center.json?version=2.346.1
12:39:03  Cache miss for: update-center-2.346.1
12:39:03  Unable to retrieve JSON from https://updates.jenkins.io/update-center.json?version=2.346.1: updates.jenkins.io: Temporary failure in name resolution
12:39:03  Unable to retrieve JSON from https://updates.jenkins.io/update-center.json?version=2.346.1: updates.jenkins.io
12:39:03  io.jenkins.tools.pluginmanager.impl.UpdateCenterInfoRetrievalException: Error getting update center json
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.getJson(PluginManager.java:810)
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.getUCJson(PluginManager.java:832)
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:217)
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:181)
12:39:03    at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:70)
12:39:03  Caused by: java.io.IOException: Unable to retrieve JSON from https://updates.jenkins.io/update-center.json?version=2.346.1: updates.jenkins.io
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.getViaHttpWithResponseHandler(PluginManager.java:1344)
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.getJson(PluginManager.java:796)
12:39:03    ... 4 more
12:39:03  Caused by: java.net.UnknownHostException: updates.jenkins.io
12:39:03    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
12:39:03    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
12:39:03    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1367)
12:39:03    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1301)
12:39:03    at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
12:39:03    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
12:39:03    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
12:39:03    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
12:39:03    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
12:39:03    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
12:39:03    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
12:39:03    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
12:39:03    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
12:39:03    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
12:39:03    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
12:39:03    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
12:39:03    at io.jenkins.tools.pluginmanager.impl.PluginManager.getViaHttpWithResponseHandler(PluginManager.java:1336)
12:39:03    ... 5 more
12:39:03  Error getting update center json
12:39:05  The command '/bin/sh -c jenkins-plugin-cli --verbose --plugin-file /usr/share/jenkins/ref/plugins.txt' returned a non-zero code: 1
  • Jenkins pipeline runs docker build in sh step:

rc = sh(returnStatus: true, script: "docker build -t ${REGISTRY}/${REPO_NAME}/${NAME} .")

  • It seems like the issue is dns issue. More specifically, failure to resolve updates.jenkins.io inside intermediate container which is running during specific docker build stage in docker-daemon container

So I ran the image in the same k8s cluster and tried to resolve updates.jenkins.io locally inside the container, it worked:

kubectl run --image=[...]  --privileged=true dind
kubectl exec -it pod/dind sh
/ # nslookup updates.jenkins.io
...

Non-authoritative answer:
Name:   updates.jenkins.io
Address: 52.202.51.185
...
  • I then tried to replicate the failing use case (container running inside docker-daemon container and updates.jenkins.io not resolving inside it) and the resolution worked:
kubectl run --image=[...]  --privileged=true dind
kubectl exec -it pod/dind sh
docker run -it alpine sh
/ # nslookup updates.jenkins.io
...

Non-authoritative answer:
Name:   updates.jenkins.io
Address: 52.202.51.185
...

Not sure why then the dns issue pops during docker build.

docker-in-docker docker-build kubernetes docker jenkins
0 Answers
Related