Server Unreachable SonarQube Webhook to Jenkins

Viewed 975

We have got two servers, one running Jenkins inside a Docker Swarm and another running SonarQube. For the last few years, Jenkins pipelines successfully triggered Sonar scans and webhooks reported back to Jenkins.

But after a SSL certificate renewal(on an ALB of the swarm) on the Jenkins side, the webhooks just constantly say "Server Unreachable". The URL for jenkins has not been changed at all.

enter image description here

And on the jenkins side, the log just waits for the quality gate to timeout.

SonarQube task 'AXz95fbXYt_5WKK0bZ00' status is 'IN_PROGRESS'
Cancelling nested steps due to timeout

The Jenkinsfile looks something like this in the quality gate stage:

timeout(time: 5, unit: 'MINUTES') {                    
         waitForQualityGate abortPipeline: true
     }
1 Answers

As I mentioned in the comment, we ran into almost exactly the same sequence of events. On the day they reconfigured SSL on our Jenkins instance, our webhooks from SonarQube to Jenkins started failing. That was more than a year ago. We've never found any clue to why this is happening.

At some point, I came to the conclusion that this would never be fixed, so I decided to figure out a workaround. It's helpful to understand the logic that happens in "waitForQualityGate". The first thing that it does is make a REST call to SonarQube to determine whether the background task is complete. If it is not, it goes into a wait loop for the completion of the task, which will not complete successfully.

So, the workaround is to ensure that when "waitForQualityGate" starts, we make sure that the background task is complete. There are two ways to do this. There's one strategy that is very simple, but has drawbacks, and there's a slightly more complex strategy that is smarter.

The very simple strategy is simply to add a "sleep" before "waitForQualityGate". How long you sleep really depends. If you sleep too short, some background tasks might not have completed by then, going into the loop again. If you sleep too long, then you waste time sitting there doing nothing useful.

The better strategy is to essentially use the same logic used in "waitForQualityGate", making the REST call to SonarQube to check on the status of the background task, in a retry loop, and exiting the loop when the status reports that the task is complete.

The logic for this would look something like this:

String authString = "${sonarProps['sonar.login']}:${sonarProps['sonar.password']}"
// If webhooks aren't going to work, then we have to poll for the task ourselves. First, we have to figure
// out the task id.
def reportFilePath = "target/sonar/report-task.txt"
def reportTaskFileExists = fileExists "${reportFilePath}"
if (reportTaskFileExists) {
    echo "Found report task file"
    def taskProps = readProperties file: "${reportFilePath}"
    echo "taskId[${taskProps['ceTaskId']}]"
    while (true) {
        sleep 20
        def taskStatusResult    =
            sh(returnStdout: true,
               script: "curl -s -X GET -u ${authString} \'${sonarProps['sonar.host.url']}/api/ce/task?id=${taskProps['ceTaskId']}\'")
            echo "taskStatusResult[${taskStatusResult}]"
        def taskStatus  = new JsonSlurper().parseText(taskStatusResult).task.status
        echo "taskStatus[${taskStatus}]"
        // Status can be SUCCESS, ERROR, PENDING, or IN_PROGRESS. The last two indicate it's
        // not done yet.
        if (taskStatus != "IN_PROGRESS" && taskStatus != "PENDING") {
            break;
        }
    }
}
Related