Docker supports a --cpus option which constrains a container to using a certain proportion of the host's computing resource. For example, --cpus 2.5 will allocate computing resource equivalent to approximately two and a half of the host's CPU capability. This allocation seems to be somewhat "logical" rather than physical, in that a container running eight processes but with --cpus 1.0 tends to run each process on a separate CPU but limited to approximately 12.5%. This is fine, I suppose.
The problem is that I am not able to find a way for a process within a constrained container to determine how many equivalent CPUs it has available. This is useful specifically for build processes, for example make -j <n>, but what is n? Commonly used nproc outputs the number of CPUs on the host regardless of the --cpus option value, which makes it unsuitable as it is far too large. The value of /proc/cpuinfo is also the host's view (same as nproc).
Is there a way for a contained process to determine the number of CPUs available to it?
As an example, on my 16-core host, I'd like to use docker run ... --cpus 4.0 ... and then have make -j <something> run only four processes rather than, say, sixteen.
FWIW, I'm using the Docker Executor with GitLab-CI and gitlab-runner, which is why --cpuset-cpus= isn't really helpful. The Executor should be able to allocate a number of CPUs to each job across all of the host's CPUs, rather than a specific subset. Although I could create separate Executors for, say, each explicit set of four CPUs, there's no obvious way I'm aware of for gitlab-runner to allocate these in a balanced way to incoming jobs, and I want to avoid pushing the responsibility onto the jobs themselves, perhaps using tags to select an Executor, since that would require each job to know too much about the runner.
I did consider setting a corresponding environment variable like NUM_CPUS=4 in the gitlab-runner's config.toml so that the build job can find out how many to use with make -j $NUM_CPUS, however variables in config.toml can only be set for all runners, not individual ones:
concurrent = 4
check_interval = 0
[[runners]]
name = "build"
url = "https://gitlab.com/"
executor = "docker"
environment = ["NUM_CPUS=4"]
[runners.docker]
# ...
cpus = "4" # match NUM_CPUS!
environment = ["FOO=456"]
The container's environment will see NUM_CPUS=4 but the statement to effect FOO=456 is invalid and of course no such value appears in the container's environment, which makes it impossible to vary the value between Executors.
I should add that in the near future I want to create Executors with different numbers of CPUs - for example, one with just one CPU for simple single-process jobs, and another with lots of CPUs for intensive multi-process jobs, and the GitLab-CI jobs will be able to pick these using GitLab-CI tags. I specifically want to avoid the situation where the pipeline jobs explicitly state that they are to use exactly, say, 4 CPUs, since that requires the jobs to know too much about the build environment and makes it brittle.