I want my RPM SIGNing job to be resilient so that I can still run my pipelines when one site/node is down

Viewed 12

I have an RPMSIGN job thats configured to run in 2 continents, Europe and North America. The Europe site is the master and N.A. is the failover incase EU is down. The problem is that when EU becomes unavailable, it doesnt failover to US. How can I make it resilient when 1 signing node isn't available so that I can still run my pipelines when one site is down.

Below is a snippet of my code:

stage('Process Signing') { when { environment name: 'IS_ARTIFACT_SIGNED', value: 'N' } // When tools{ jdk jenkinsToolkits.LoadJdkTool("${buildConfig.JAVA_VERSION}") maven jenkinsToolkits.LoadMavenTool("${buildConfig.MAVEN_VERSION}") } // Tools steps { script{ String BUILD_NODE String ARTIFACT_BUILD_HOST String[] RPM_LIST = readFile("${JOB_BASE_NAME}_${BUILD_NUMBER}.list").split('\n') for (String RPM_ITEM in RPM_LIST){ ARTIFACT_BUILD_HOST = rpmSign.ReadRpmBuildHost(rpmSign.ReadRpmInfo(pwd(), "${RPM_ITEM}")) try { echo "Trying EU jenkins agent for signing" BUILD_HOST = "eu01" // Valid eu01, eu02, eu04 echo "${BUILD_HOST}" BUILD_NODE = "${BUILD_HOST}_build" stash includes: "${RPM_ITEM}", name: "${RPM_ITEM}-${BUILD_NUMBER}"

                            SIGNING_SERVER = rpmSign.GetSigningServer("${BUILD_HOST}")

                            node ("${BUILD_NODE}"){
                                //sh "echo $PATH"
                                unstash "${RPM_ITEM}-${BUILD_NUMBER}"
                                rpmSign.ApplyRpmSigning("${SIGNING_SERVER}", "${BUILD_HOST}", "${RPM_ITEM}")
                                stash includes: "${RPM_ITEM}", name: "${RPM_ITEM}-${BUILD_NUMBER}-SIGNED"
                                cleanWs()
                            } // Node
                        } // Try
                        catch(err) {
                            echo "EU jenkins agent failed....trying US jenkins agent"
                            BUILD_HOST = "us01" // Valid us01, us02, us03
                            echo "${BUILD_HOST}"
                            BUILD_NODE = "${BUILD_HOST}_build"
                            stash includes: "${RPM_ITEM}", name: "${RPM_ITEM}-${BUILD_NUMBER}"

                            SIGNING_SERVER = rpmSign.GetSigningServer("${BUILD_HOST}")

                            node ("${BUILD_NODE}"){
                                //sh "echo $PATH"
                                unstash "${RPM_ITEM}-${BUILD_NUMBER}"
                                rpmSign.ApplyRpmSigning("${SIGNING_SERVER}", "${BUILD_HOST}", "${RPM_ITEM}")
                                stash includes: "${RPM_ITEM}", name: "${RPM_ITEM}-${BUILD_NUMBER}-SIGNED"
                                cleanWs()
                            } // Node
                        } // Catch
                        
               
0 Answers
Related