How to define what are the components of a heterogeneous job?

user_8473437 0
Viewed 66

I am trying to learn how to use heterogeneous jobs with Slurm. I've been reading this resource on their official website but I really don't understand how the setup should look like. I understand that heterogeneous jobs are started with this piece of code they've provided:

$ sbatch --cpus-per-task=4 --mem-per-cpu=16g --ntasks=1 : \
         --cpus-per-task=2 --mem-per-cpu=1g  --ntasks=8 my.bash

However, what I don't understand is how do I define what is the first task and what is the second task. For example, let's say that I have two R scripts in the same folder called script_1.R and script_2.R I want to submit them to some cluster via Slurm as a heterogeneous job, and I want to request 16 GB for the first one and 1 GB for the second one. Where is the part of the script in sbatch where I provide the path to the files for each task? Or is it done somewhere outside of the sbatch? How does this work? Any help would be appreciated, thanks!

slurm
1 Answers

As far as I understand it, the tasks have to be specified in the script (in your example my.bash). There, the tasks have to be submitted to the srun command.

I think in your example it should look somewhat like the following:

#!/bin/bash
srun Rscript path/to/script_1.R : Rscript path/to/script_2.R

Disclaimer: I have no idea if this works. It's just what I figured after reading the documentation of srun and the heterogenous job documentation.

user_169858820
Related