I have a snakemake workflow I would like to run on GKE. My jobs are written to run in a singularity container, the image for which I am storing in a private Google Artifact Registry. The problem I am running into is how to ensure my snakemake jobs have persistent permission to pull this container.
Things I have tried:
- Passing an oauth token to the job using the
SINGULARITY_DOCKER_USERNAME,SINGULARITY_DOCKER_PASSWORDenvvars. This works for the first hour until the token lapses. GCP does not allow for longer lasting tokens or persistant username/passwords afaik. - I have set up workflow identity on the cluster with a google service account that has permissions to the registry. I still get a
403: forbiddenwhen I try to pull from the registry.
Has anyone gotten snakemake on k8s to work with google artifact registry? If so how? Thanks