Using find and cat to duplicate the content of multiple files

user_3133075 0
Viewed 181

Dears,

I have tried a combination of "find" and "cat" in a bash script to duplicate contents of hundreds of files in sub-directories:

find ./inlet3/* -type f -name 'U' -exec bash -c 'cat "{}"' \; > temp -exec bash -c 'cat temp >> "{}"' \;

Looking at the given command, I intend to first copy the content of each "U" into a temporary file named "temp", which is supposed to be created each time in the directory where the bash script exists. This part should be achieved by the following method:

-exec bash -c 'cat "{}"' \; > temp

The content of "temp" should be subsequently appended to "U", to have a file with duplicated content. This part is supposed to be done by the given method:

-exec bash -c 'cat temp >> "{}"' \;

However, the command does not give me the desired result. I would anticipate that for each "U" found via find, "temp" would be overwritten. But I see that the content of previously found "U"s are appended to "temp", accumulating its content. If I try to remove "temp" with the following method:

-exec rm -rf temp \;

then I will receive an error:

cat: temp: No such file or directory

There are some other -exec options that I have omitted for simplicity.

Could anybody say where I might be wrong? I appreciate any hint.

Kind regards.

cat duplicates find bash
2 Answers

You want -execdir instead of -exec to create the temp file in the same directory as U. And the redirection should be in the bash session: in fact all the shell operations can be run in the same shell:

find ... -execdir sh -c 'temp=$(mktemp) && cat "{}" > "$temp" && cat "$temp" >> "{}" && rm "$temp"'

If you install the moreutils package, you can use sponge without needing to create a temp file:

find ... -execdir sh -c '{ cat "{}"; cat "{}"; } | sponge "{}"'
user_75520

Traditionally, we say

echo foo > bar

echo sees foo as it's argument, and sends it to stdout, which we have redirected to bar... but it doesn't have to be written this way. In fact, echo doesn't even know its output has been redirected. That's the parser handling the streams, so the result means you can write the command in some weird ways -

echo > bar foo

also works, as done

> bar echo foo

This is counter-intuitive, but nonetheless true.

Your command was

find ./inlet3/* -type f -name 'U' -exec bash -c 'cat "{}"' \; > temp -exec bash -c 'cat temp >> "{}"' \;

This is functionally the same as

find ./inlet3/* -type f -name 'U' -exec bash -c 'cat "{}"' \; -exec bash -c 'cat temp >> "{}"' \;  > temp

or

> temp find ./inlet3/* -type f -name 'U' -exec bash -c 'cat "{}"' \; -exec bash -c 'cat temp >> "{}"' \;

Accordingly, all output from the find is going into temp.

What you wanted was

find ./inlet3/* -type f -name 'U' -exec bash -c 'cat "{} >| temp"' \; -exec bash -c 'cat temp >> "{}"' \;

Unless you set -o noclobber the result of > is functionally identical to >|, but I tend to be explicit when I want truncation. It never hurts.

As Glenn Jackman pointed out, fewer subshells is also better, so -

find ./inlet3/* -type f -name 'U' -exec bash -c 'cat "{}" >| temp && cat temp >> "{}"' \;
user_86565520
Related