slurm --array two variables

user_7249029 0
Viewed 88

I have the following bash script that runs on a HPC using slurm:

#!/bin/bash
#SBATCH --job-name test
#SBATCH --ntasks 10
#SBATCH --time 00-01:00
#SBATCH --output out
#SBATCH --error err
#SBATCH --array=0-9

readarray -t VARS < list_VAR.txt
VAR=${VARS[$SLURM_ARRAY_TASK_ID]}
export VAR

bash data_0_"$VAR".sh

The above bash script sends 10 jobs (#SBATCH --array=0-9) to the HPC to run the data_0_"$VAR".sh script, where "$VAR" is a given string contained within the list_VAR.txt file.

Let's suppose now I have a second list_VAR_2.txt file that contains a list of numbers from 0 to 3 and I want to apply it to the job-array above, along with list_VAR.txt. The data_0_"$VAR".sh script to be ran will then convert to data_"$VAR_2"_"$VAR".sh.

Is there a way to add this further list of variables list_VAR_2.txt to the bash script?

Thanks

#####################

Update, list_VAR.txt

aa
bh
wwe
ftq
juu
d
8i
yz5
qq1p
m75

list_VAR_2.txt

0
1
2
3
script slurm hpc bash
1 Answers

You could load your list_VAR_2.txt content into an array like you did for the first file. Then loop on the arrays to build your bash commands.

Ex:

#!/bin/bash

readarray -t VARS < list_VAR.txt
readarray -t VARS_2 < list_VAR_2.txt

for VAR in "${VARS[@]}"
do
    for VAR_2 in "${VARS_2[@]}"
    do
        bash data_"$VAR_2"_"$VAR".sh
    done
done

Or build the bash commands by specifying the indexes of the array elements you want.

Ex.

bash data_"${VARS[INDEX1]}"_"${VARS_2[INDEX2]}".sh
user_23209610
Related