How to run for loop in batch of 4 for 20 items at once in jenkinsfile

user_11417612 0
Viewed 1130

I have to run for loop in groovy for 40 items, but do wish to run it for 4 items in parallel then next batch and so on. I know of parallel deployments in jenkinsfile but it triggers all 40 at once.

          def i = 0
          mslist.collate(4).each{
               build job: 'deploy', parameters: [string(name: 'PROJECT', value: "${it[i]}"), string(name: 'ENVIRONMENT', value: params.ENVIRONMENT)]
               i=i+1
}

My Updated code:

stages {
    stage ('Parallel Deployments') {
      steps {
        script {
          def m = rc.ic()
          m = m.collect { "${it}" }
          println "$m"
          m.collate(4).each{
              def deployments = [:]
              batch.each {
                deployments[it] = {
                   build job: 'jb', parameters: [string(name: 'pt', value: it), string(name: 'pl', value: params.gh), string(name: 'dc', value: params.nb)]
                }   
              
            }  
            parallel deployments
          }
        deployments["failFast"] = false
        }
    }
  } 
 }
jenkins-groovy jenkins-pipeline jenkins groovy
2 Answers

It can be done like this :

node {
    def items = (1..40).collect { "item-${it}" }
    items.collate(4).each { List batch -> 
        def n=[:]
        
        batch.each {
            n[it] = {
                stage(it) {
                    build job: 'x', parameters: [ string( name: "it", value : it) ]      
                }    
            }
        }
        
      parallel n
    }
}

job: x Jenkinsfile content

node {
    echo "Hello from Pipeline x"
    print params
}

This will invoke 4 jobs at a time and run parallelly. Make sure you have more than 4 executors configured on Jenkins.

enter image description here

user_47525160

You can do something like:

def items = (1..40).collect { "item-${it}" }
items.collate(4).each { List batch -> 
  // batch is now a list of 4 items, do something with it here
}

to use the groovy Iterable.collate method to split the items into batches of four items and loop through the batches.

If you really want to do this "in parallell" as in using multiple threads, then that is a different question.

user_54494310
Related