Limit cpu's with Jenkins and docker plugin

Question

I've got a Jenkins setup using Docker containers as build slaves.

The containers are provisioned as agents via the Docker plugin - rather than the Jenkinsfiles instantiating the container.

Machine is an 8 thread i7 with 16GB ram and 16GB swapfile. Currently I'm running just a single container.

If the build in the container uses all 8 threads, it seems to cause OS OOM kills of some of the GCC processes when it is building. There is a mix of build tasks, some I can explicitly control the number of make threads, others it will query the system for the number of cores and go wide. When there are the maximum number of threads active it appears to run out of RAM without throttling or stalling the work - it also doesn't seem to make full use of the swap.

I want to limit the number of CPU cores the Docker slave is allowed to use but I can't find a way to pass the --cpus=2 argument to the Docker run command. the shares argument doesn't appear to have the effect I want.

I'm happy to divvy up the resources explicitly from the container configuration in order to make the server more reliable, but I don't want it running into hard limits and getting OOM killed.

user_11024239 · Answer

I am not quite sure how you start the Docker container but if you do it with help of the plug-in, I am guessing that you make a call in your Jenkinsfile like myImage.inside("..."). In that case you should be able to add "--cpus=2" as part of the string argument. This works at least for us.

The drawback with this solution is however that all Jenkins jobs have to be updated and you can never be sure that all creators of Jenkins jobs remember this. Hence we implemented a different solution which should work for you if the above solution does not. We basically created a script to work as a wrapper when anyone called docker. It injects "--cpus=2" if the "docker run" is called. Otherwise it just forwards the call to Docker. We accomplished this by:

Put the following script in /home/jenkins/bin/docker . ("docker" being the file, not a folder)

    #!/bin/bash

    echo COMMAND: $0 with arguments: $@ >&2

    if [ "$1" = 'run' ] ; then
            echo 'INFO: I found a "run" command, limiting to 2 cpus' >&2

            echo RUNNING: /usr/bin/docker "$1" --cpus=2 "${@:2}" >&2
            /usr/bin/docker "$1" --cpus=2 "${@:2}"
    else
            echo RUNNING: /usr/bin/docker "$@" >&2
            /usr/bin/docker "$@"
    fi

Make it executable

    chmod a+x /home/jenkins/bin/docker

Modify the path by updating ~/.bashrc . (~/.bash_profile only takes affect when you log in, meaning that Jenkins will not pick it up.) Put "/home/jenkins/bin" FIRST in the path, e. g.

    PATH=/home/jenkins/bin:$PATH

Limit cpu's with Jenkins and docker plugin

1 Answers