Matching file content with other filenames to extract and merge contents

user_6671100 0
Viewed 72

I have two directories.

In directory_1, I have many .txt files

Content of these files (for example file1.txt) are a list of characters

file1.txt 
--

rer_098
dfrkk9

In directory_2, I have many files, two of them are ‘rer_098’ and ‘dfrkk9’.

Content of these files are as follows:

rer_098
--

>123_nbd
sasert
>456_nbd
ffjko

dfrkk9
--

>789_nbd
figyi
>012_nbd
jjjygk

Now in a separate output directory (directory_3), for this above example, I want output files like:

file1.txt
--

>123_nbd
sasert
>456_nbd
ffjko
>789_nbd
figyi
>012_nbd
jjjygk

and so on for file2.txt

Thanks!

sed awk linux
3 Answers

This might work for you (GNU parallel):

parallel 'cat {} | parallel -I@@ cat dir_2/@@ > dir_3/{/}' ::: dir_1/*.txt

Use two invocations of parallel, the first traverses dir_1 and pipes its output in a second parallel. This cats the input files and outputs the result dir_3 keeping the original name from the first parallel invocation.

N.B. The use of the -I option to rename the parameter delimiters from the default {} to @@.

user_9674920

Pretty easy to do with just shell. Something like

for fullname in directory_1/*.txt; do
    file=$(basename "$fullname")
    while read -r line; do
        cat "directory_2/$line"
    done <"$fullname" >"directory_3/$file"
 done
user_99521960 
for file in directory_1/*.txt; do
    awk 'NR==FNR{ARGV[ARGC++]="directory_2/"$0; next} 1' "$file" > "directory_3/${file##%/}"
done
user_17450010
Related