How do I make condor name my files as follow:

meta_learning_experiments_submission.py.e451863 meta_learning_experiments_submission.py.o444375

$(FILENAME).e$(CLUSTER)
$(FILENAME).e$(CLUSTER)

I tried it but it doesn't seem to work.

e.g. so that it matches the default behaviour of PBS when I do qsub?

I also like to know how to make slurm also match it, it would be perfect (so all three clusters are working).

I tried:

#SBATCH --error="(filename).%j.%N.err"

but that didn't work

Bounty

I want to set the file name automatically without hardcoding it.

related:

• https://www.quora.com/unanswered/How-do-I-make-a-current-file-the-extension-with-condor-match-output-error-files-to-have-PBS-and-Slurm-have-the-same-output-files

For context here is my current submit file:

####################
#
# Experiments script
# Simple HTCondor submit description file
#
#
# chmod a+x test_condor.py
# chmod a+x experiments_meta_model_optimization.py
# chmod a+x meta_learning_experiments_submission.py
# chmod a+x download_miniImagenet.py
# chmod a+x ~/meta-learning-lstm-pytorch/main.py
# chmod a+x /home/miranda9/automl-meta-learning/automl-proj/meta_learning/datasets/rand_fc_nn_vec_mu_ls_gen.py
# chmod a+x /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/supervised_experiments_submission.py
# chmod a+x /home/miranda9/automl-meta-learning/results_plots/is_rapid_learning_real.py
# condor_submit -i
# condor_submit job.sub
#
####################

Path = /home/miranda9/automl-meta-learning/
Path = /home/miranda9/ML4Coq/

# Executable = /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/supervised_experiments_submission.py
# Executable = /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/meta_learning_experiments_submission.py
# Executable = /home/miranda9/meta-learning-lstm-pytorch/main.py
# Executable = /home/miranda9/automl-meta-learning/automl-proj/meta_learning/datasets/rand_fc_nn_vec_mu_ls_gen.py
# Executable = /home/miranda9/automl-meta-learning/results_plots/is_rapid_learning_real.py

## Output Files
Log          = experiment_output_job.$(CLUSTER).log.out
Output       = experiment_output_job.$(CLUSTER).out.out
Error        = experiment_output_job.$(CLUSTER).err.out

Output = %(FILENAME).o$(CLUSTER)

# Use this to make sure 1 gpu is available. The key words are case insensitive.
# REquest_gpus = 1
# requirements = (CUDADeviceName != "Tesla K40m")
# requirements = (CUDADeviceName == "Quadro RTX 6000")

# requirements = ((CUDADeviceName = "Tesla K40m")) && (TARGET.Arch == "X86_64") && (TARGET.OpSys == "LINUX") && (TARGET.Disk >= RequestDisk) && (TARGET.Memory >= RequestMemory) && (TARGET.Cpus >= RequestCpus) && (TARGET.gpus >= Requestgpus) && ((TARGET.FileSystemDomain == MY.FileSystemDomain) || (TARGET.HasFileTransfer))
# requirements = (CUDADeviceName == "Tesla K40m")
# requirements = (CUDADeviceName == "GeForce GTX TITAN X")

# Note: to use multiple CPUs instead of the default (one CPU), use request_cpus as well
# Request_cpus = 4
Request_cpus = 16

# E-mail option
Notify_user = me@gmail.com
Notification = always

Environment = MY_CONDOR_JOB_ID= $(CLUSTER)

# "Queue" means add the setup until this line to the queue (needs to be at the end of script).
Queue

Alternatively, if I can use my executable script as the submission script with the parameters at the top that would work too

I usually use my submission script be the same as my executable script, e.g. see my qsub example:

#!/homes/miranda9/.conda/envs/automl-meta-learning/bin/python
#PBS -V
#PBS -M mee@gmail.com
#PBS -m abe
#PBS -lselect=1:ncpus=112

import sys
import os

for p in sys.path:
    print(p)

print(os.environ)