Can I cancel just the array job in slurm?

user_402668 0
Viewed 1050

I have a bunch of jobs running as an array job in slurm:

123_[1-500] PD my_job 0:00 me
123_2       R  my_job 9:99 me
123_3       R  my_job 9:99 me
123_4       R  my_job 9:99 me
123_5       R  my_job 9:99 me
...

As I read the man page on scancel, it seems to indicate that if I execute scancel 123 it will stop everything

Am I wrong, or is there another way to stop just the array job? I want the already running jobs to finish, I just don't want any more jobs created by 123, and I really don't want to have to figure out which jobs need to be re-run if I accidentally kill them mid-way

slurm
2 Answers

You can issue scancel with the additional --state tag:

 scancel --state=PENDING 123

or, in short:

 scancel -t PD 123

That will only cancel jobs of the 123 array that are pending and will leave the running the already started ones.

user_17636140

I put a hold on the job: scontrol hold 123 Once all the jobs reported by squeue were done, I was able to cancel it.

user_4026680
Related