Limit the docker containers executed in a loop

user_4638699 0
Viewed 123

I am very new to docker, have created a Dockerfile to create an image that executes the protractor tests.

That Dockerfile has an entry point that expects a parameter with Suite Name that I want to execute.

It all runs very well when I provide suite names in the command line. I have about 30 test suites, I am using another .sh file which filters out suite names, and in a for loop, it runs docker commands with different suite names.

Now I do not want to sun 30 suites simultaneously but want to set a limit of say 6 at a time and want to keep others waiting until one is finished.

I execute like this:

for (( i=0; i<${tests}; i++ ));
do
  docker run -dit containername $testSuiteName
done

So how can I limit the maximum number of executions at a time?

protractor docker sh
2 Answers

There are going to be a number of ways of tackling this problem. Here is one possible solution.

You can treat this as a shell scripting problem, rather than a Docker problem. For example, consider the following, which instead of docker run ... just uses sleep:

#!/bin/sh

let count=5
let tests=20

for (( i=0; i<tests; i++ )); do
  sleep $((RANDOM % 10)) &
  echo "started $!"
  let count--
  if (( count <= 0 )); then
    wait -n
    let count++
  fi
done

echo "waiting for remaining jobs"
wait

echo "all done"

This starts $count processes in parallel, and then waits for one to exit. When a process exits, it immediately starts a new one. Once it has started all the jobs, it simply waits for everything to finish.

Using this model, you would drop the -d from your docker run command line, since you need the shell to track the background processes. Instead of sleep ... &, you would run:

docker run containername $testSuiteName > /dev/null 2>&1 &

Note that I've dropped the -i and -t options here, since it looks like you're running these tests non-interactively.

user_1473560

A few more possible solutions:

  1. If you run your tests in jenkins, you can create a job that runs one test suite. Set the max number of executors to 6. And start as many tests as you want. Now jenkins won't let run more than 6 jobs at a time.

  2. This is I think the ideal and most correct approach, yet most difficult one. You can use an orchestrator as kubernetes. This actually controls all your docker image. Unfortunately, I don't have step by step guide how to achieve. But this is really the most professional way to tackle your problem

user_91501460
Related