Arrays and user job limitations on SLURM?

user_14144132 0
Viewed 188

I'm a new SLURM user and I'm trying to figure out the best way to submit a job that requires the same command to run 400,000 times with different input files (approximately 200MB memory per CPU, 4 minutes for one instance, each instance runs independently).

I read through the documentation, and so far it seems that arrays are the way to go.

I can use up to 3 nodes on my HPC with 20 cores each, which means that I could run 60 instances of my command at the same time. However, user limit for jobs running at the same time is 10 jobs, with 20 jobs in the queue.

So far, everything I've tried runs each instance of the command as a separate job, thus limiting it to 10 instances in parallel.

How can I fully utilize all available cores in light of the job limits?

Thanks in advance for your help!

slurm bash
2 Answers

You can have a look at tools like GREASY that will allow you to run a single Slurm job and spawn multiple subtasks.

The documentation specifies how to install it and use it and can be found here

user_24719630

You don't even need the job array to attain the defined objective. Firstly submit a job via sbatch job_script command, in the job_script you can customise the job submission. You can use srun parameters & along with the for loop to run the maximum jobs.

user_109119320
Related