How to process a list of files with SLURM

Viewed 3187

I'm new to SLURM. I want to process a list of files assembled_reads/*.sorted.bam in parallel. With the code below, however only one process is being used over and over again.

#!/bin/bash
#
#SBATCH --job-name=****
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=24
#SBATCH --partition=short
#SBATCH --time=12:00:00
#SBATCH --array=1-100
#SBATCH --mem-per-cpu=16000
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=****@***.edu
srun hostname

for FILE in assembled_reads/*.sorted.bam; do
  echo ${FILE}
  OUTFILE=$(basename ${FILE} .sorted.bam).raw.snps.indels.g.vcf
  PLDY=$(awk -F "," '$1=="$FILE"{print $4}' metadata.csv)
  PLDYNUM=$( [[$PLDY = "haploid" ]] && echo "1" || echo "2")

  srun java -Djava.io.tmpdir="tmp" -jar GenomeAnalysisTK.jar \
  -R scaffs_HAPSgracilaria92_50REF.fasta \
  -T HaplotypeCaller \
  -I ${${SLURM_ARRAY_TASK_ID}} \
  --emitRefConfidence GVCF \
  -ploidy $PLDYNUM \
  -nt 1 \
  -nct 24 \
  -o $OUTFILE
  sleep 1 # pause to be kind to the scheduler
done
1 Answers
Related