In Slurm, is there a quick command to determine the total number of jobs (pending and active) at a given moment?

user_7498328 0
Viewed 6131

In slurm, calling the command squeue -u <username> will list all the jobs that are pending or active for a given user. I am wondering if there was a quick way to tally them all so that I know how many outstanding jobs there are, including pending and actively running jobs. Thanks!

slurm linux bash
3 Answers

I would interprete "quick command" differently. Additionally I would add -r for cases when you are using job arrays:

squeue -u <username> -h -t pending,running -r | wc -l

option -h removes the header "wc -l" (word count) counts the line of the output. Eventually I am using it with watch

watch 'squeue -u <username> -h -t pending,running -r | wc -l'
user_127699850

If you just want to summarize the output of squeue, how about:

squeue -u <username> | awk '
BEGIN {
    abbrev["R"]="(Running)"
    abbrev["PD"]="(Pending)"
    abbrev["CG"]="(Completing)"
    abbrev["F"]="(Failed)"
}
NR>1 {a[$5]++}
END {
    for (i in a) {
        printf "%-2s %-12s %d\n", i, abbrev[i], a[i]
    }
}'

which yields something like:

R  (Running)    1
PD (Pending)    4

Explanations:

  • The job state is assumed to be in the 5th field according to the default format of squeue.
  • Then the script counts the appearance of each job state code except for the 1st line which includes the header.
  • Finally it reports the count of each job state code.

In order to make it handy, add the following lines to your .bash_aliases or .bashrc (the filename may depend on the system):

function summary() {
    squeue "$@" | awk '
    BEGIN {
        abbrev["R"]="(Running)"
        abbrev["PD"]="(Pending)"
        abbrev["CG"]="(Completing)"
        abbrev["F"]="(Failed)"
    }
    NR>1 {a[$5]++}
    END {
        for (i in a) {
            printf "%-2s %-12s %d\n", i, abbrev[i], a[i]
        }
    }'
}

Then you can invoke the command just with summary [option], where [option] accepts options to squeue if needed (mostly unnecessary).

Hope this helps.

user_85723800

This comment from @skytaker really provides the best solution in my view, distinguishing between pending and running, giving you a separate count for each. Only difference to Stefan Maschek's solution is the use of uniq as opposed to wc:

squeue -u <username> -h -t pending,running -r -O "state" | uniq -c
user_74832110
Related