Run parallel commands in script on Jenkins

Estack 0
Viewed 20

I have a python script in which i run two commands using subProcess. I want these commands to run in parallel. When i run the python script on my machine these commands do run in parallel but when i run the script via Jenkins it runs them one after the other. How can i run these command in parallel in jenkins?

    reg_path = os.environ['PWD']
    command1 = f'python3 {python_script_path} {reg_path} '
    command2 = f'python3 {python_script_path} {reg_path} '

    processes = []

    for i in range(1):
        f = subprocess.Popen(command2, shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        p = subprocess.Popen(command1, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        processes.append((p, f))
        p.wait()
        out, err = p.communicate()
        fout, ferr = f.communicate()
        p.kill()
        f.kill()
       
        errMsg = 'other processes are running'
        assert errMsg in str(fout) or errMsg in str(out)

Thanks

jenkins python
1 Answers

On Jenkins I would suggest you use the parallel pipeline if possible:

pipeline {
    agent any
    stages {
        stage("my python scripts") {
            parallel {
                stage("Script 1") {
                     sh("python3 script1.py ${env.WORKSPACE}")
                }
                stage("Script 2") {
                     sh("python3 script2.py ${env.WORKSPACE}")
                }
            }
        }
    }
}

This will run both your script1.py and script2.py in parallel, without requiring a managing script to handle subprocesses.

lkraider0
Related