force GNU-make to use modification time of symbolic link target, not modification time of the source

user_897521 0
Viewed 796

I am experiencing a problem with modification times in GNU make 3.81.

My Makefile uses data mounted from a remote server which, for some reason unknown to me, has set the Modification Time of the files set into the far future (into the next century).

The Makefile first creates a symbolic link to the remote file "from the future" in a local directory and then based on this file runs several scripts which all generate some output files.

Now, when I want to rerun "make all" after it was interruped, to get all my output files, it will not restart with the latest output file it had generated (let's say from script number 3), but instead, it regenerates everything from start, since it notices that the first file (the one where I symlinked the file "from the future") is newer.

Is there an option to tell make to take the modification time of the symlink itself, and not the modification time of the symlink's target?

Example

Here is a minimal working example that regenerates the problem:

Set up folder and files:

mkdir symlinkmake
cd symlinkmake
echo $PWD > futurefile.txt
# set file modification time to the future
touch -t 212111111111 futurefile.txt

Content of the Makefile:

all: symlink.txt first_output.txt second_output.txt

symlink.txt:
    ln -s futurefile.txt symlink.txt

first_output.txt: symlink.txt
    cut -f1 -d"/" symlink.txt > first_output.txt

second_output.txt: first_output.txt
    wc first_output.txt > second_output.txt

Now run make all and delete the second output file. Then rerun make. Since the first output file already exists, only the second file needs to be generated. But as futurefile.txt is newer than any other file, the first output file will also be generated:

make all
rm second_output.txt
make all

On my machine the output looks like this:

$ ls
Makefile        futurefile.txt  symlink.txt
$ make all
make: Warning: File `symlink.txt' has modification time 3.3e+09 s in the future
cut -f1 -d"/" symlink.txt > first_output.txt
wc first_output.txt > second_output.txt
make: warning:  Clock skew detected.  Your build may be incomplete.
$ rm second_output.txt
$ make all
make: Warning: File `symlink.txt' has modification time 3.3e+09 s in the future
cut -f1 -d"/" symlink.txt > first_output.txt
wc first_output.txt > second_output.txt
make: warning:  Clock skew detected.  Your build may be incomplete.
gnu-make symlink unix makefile
1 Answers

There is the --check-symlink-times make option but:

On systems that support symbolic links, this option causes make to consider the timestamps on any symbolic links in addition to the timestamp on the file referenced by those links. When this option is provided, the most recent timestamp among the file and the symbolic links is taken as the modification time for this target file.

This is not what you want which is to consider only the link's timestamp. As you cannot fix your date/time problem, I can imagine only one 2 solutions

  1. order-only prerequisites (OOP):

    .PHONY: all

all: first_output.txt second_output.txt | symlink.txt

symlink.txt:
    ln -s futurefile.txt $@

first_output.txt: | symlink.txt
    cut -f1 -d"/" $| > $@

second_output.txt: first_output.txt
    wc $< > $@

    symlink.txt being an OOP make will only consider its existence, not its timestamp. In other words, make will build it, and (re-)build all the targets that depend on it, only if it is missing.

    Note: I also used automatic variables everywhere it was possible. They are handy, less error prone and frequently allow to factorize rules.

    Drawback: this is more a quick and dirty hack than a real solution. Indeed, if the file referenced by the link changes, your other targets will not be rebuilt:

    $ ls -al
Sep 14 14:55 Makefile
Dec  1  2018 futurefile.txt
$ make
ln -s futurefile.txt symlink.txt
cut -f1 -d"/" symlink.txt > first_output.txt
wc first_output.txt > second_output.txt
$ ls -al
Sep 14 14:55 Makefile
Sep 14 15:05 first_output.txt
Dec  1  2018 futurefile.txt
Sep 14 15:05 second_output.txt
Sep 14 15:05 symlink.txt -> futurefile.txt
$ make
make: Warning: File 'symlink.txt' has modification time 6688452 s in the future
make: Nothing to be done for 'all'.
make: warning:  Clock skew detected.  Your build may be incomplete.
$ touch --date=2018-12-02 futurefile.txt
$ ls -al futurefile.txt
$ make
make: Warning: File 'symlink.txt' has modification time 6774852 s in the future
make: Nothing to be done for 'all'.
make: warning:  Clock skew detected.  Your build may be incomplete.

  2. To mitigate the drawback of the first solution, you will have replace the date/time of your local host by the date/time of your remote server when building your targets. By chance, the touch command has a nice --reference option that force the timestamp of a file to be that of another file:

    .PHONY: all

all: first_output.txt second_output.txt

first_output.txt: futurefile.txt
    cut -f1 -d"/" $< > $@
    touch --reference=$< $@

second_output.txt: first_output.txt
   wc $< > $@
   touch --reference=$< $@

    Drawback: instead of one symbolic link, you will have two files (first_output.txt and second_output.txt) with time in the future:

    $ ls -al
Sep 14 14:55 Makefile
Dec  1  2018 futurefile.txt
$ make
make: Warning: File 'futurefile.txt' has modification time 6688320 s in the future
cut -f1 -d"/" futurefile.txt > first_output.txt
touch --reference=futurefile.txt first_output.txt
wc first_output.txt > second_output.txt
touch --reference=first_output.txt second_output.txt
make: warning:  Clock skew detected.  Your build may be incomplete.
$ ls -al
Sep 14 14:55 Makefile
Dec  1  2018 first_output.txt
Dec  1  2018 futurefile.txt
Dec  1  2018 second_output.txt
$ make
make: Warning: File 'first_output.txt' has modification time 6688320 s in the future
make: Nothing to be done for 'all'.
make: warning:  Clock skew detected.  Your build may be incomplete.
$ touch --date=2018-12-02 futurefile.txt
$ ls -al futurefile.txt
$ make
make: Warning: File 'first_output.txt' has modification time 6688320 s in the future
cut -f1 -d"/" futurefile.txt > first_output.txt
touch --reference=futurefile.txt first_output.txt
wc first_output.txt > second_output.txt
touch --reference=first_output.txt second_output.txt
make: warning:  Clock skew detected.  Your build may be incomplete.
user_17737980
Related