snakemake warning that code used to generate one or several output files has changed, but they haven't

user_4392395 0
Viewed 242

I'm running into an issue where when I dry-run (or run for real) a rule I get this message...

The code used to generate one or several output files has changed:
    To inspect which output files have changes, run 'snakemake --list-code-changes'.
    To trigger a re-run, use 'snakemake -R $(snakemake --list-code-changes)'.

When I look at the files that have changes, their time stamps are newer than the code used to generate them and the files have time stamps that are in the correct order. Some of these files take a long time to generate and I'd really rather not have to recreate them.

Also, if the files have changed, then why wouldn't the DAG indicate that they need to be created? The files that supposedly have changes aren't getting updated the next time I run a rule.

googling the error message hasn't gotten me very far.

As an example, snakemake --list-code-changes includes data/references/silva.v4.align and data/references/silva.v4.tax. Here's the rule...

rule silva:
  input:
    script="code/get_silva.sh"
  output:
    "data/references/silva.v4.align",
    "data/references/silva.v4.tax"
  resources:  
    cpus=8,
  shell:
    """
    {input.script} {resources.cpus}
    """

Here is the ls -lth output...

-rw-rw-r-- 1 pschloss schloss-lab 589K Apr  2 09:29 data/references/silva.v4.tax
-rw-rw-r-- 1 pschloss schloss-lab  53M Apr  2 09:29 data/references/silva.v4.align
-rwxrwxr-x 1 pschloss schloss-lab 1.4K Mar 29 09:52 code/get_silva.sh

Initially I was doing this with v.7.3.1 and upgraded to 7.3.8. I am still getting the message.

snakemake python
2 Answers

If you are using version before 7.3.2, consider upgrading. There was a code change detection bug and it was fixed in 7.3.2, so upgrading snakemake should remove the message you've been seeing:

pip install --upgrade snakemake
user_106935960

It says that the code has changed (the code of the snakefile itself as I understand). Is the snakefile more recent? Did you make any change?

As you say, the files and their sources have correct timestamps so actually nothing needs to be done in principle. But possibly the code of rules itself have changed so snakemake is assuming that possibly some rule should be re-run to execute the new code even if the source files haven't changed.

You can find out which rules have changed by using the suggested argument --list-code-changes. Maybe you made a recent change in the code of the snakefile after executing a rule.

user_133542800
Related