how to pipe multi commands to bash?

user_6521116 0
Viewed 102

I want to check some file on the remote website.

Here is bash command to generate commands that calculate the file md5

[root]# head -n 3 zrcpathAll | awk '{print $3}' | xargs -I {} echo wget -q -O - -i {}e \| md5sum\;
wget -q -O - -i https://example.com/zrc/3d2f0e76e04444f4ec456ef9f11289ec.zrce | md5sum;
wget -q -O - -i https://example.com/zrc/e1bd7171263adb95fb6f732864ceb556.zrce | md5sum;
wget -q -O - -i https://example.com/zrc/5300b80d194f677226c4dc6e17ba3b85.zrce | md5sum;

Then I pipe the outputed commands to bash, but only the first command was executed.

[root]# head -n 3 zrcpathAll | awk '{print $3}' | xargs -I {} echo wget -q -O - -i {}e \| md5sum\; | bash -v
wget -q -O - -i https://example.com/zrc/3d2f0e76e04444f4ec456ef9f11289ec.zrce | md5sum;
3d2f0e76e04444f4ec456ef9f11289ec  -
[root]#
xargs pipe shell bash
1 Answers

Would you please try the following instead:

while read -r _ _ url _; do
    wget -q -O - "$url"e | md5sum
done < <(head -n 3 zrcpathAll)

we should not put -i in front of "$url" here.

[Explanation about -i option]

Manpage of wget says:

-i file
--input-file=file
Read URLs from a local or external file. [snip]
If this function is used, no URLs need be present on the command line. [snip]
If the file is an external one, the document will be automatically treated as html if the Content-Type matches text/html. Furthermore, the file's location will be implicitly used as base href if none was specified.

where the file will contain line(s) of url such as:

https://example.com/zrc/3d2f0e76e04444f4ec456ef9f11289ec.zrce
https://example.com/zrc/e1bd7171263adb95fb6f732864ceb556.zrce
https://example.com/zrc/5300b80d194f677226c4dc6e17ba3b85.zrce

Whereas if we use the option as -i url, wget first downloads the url as a file which contains the lines of urls as above. In our case, the url is the target to download itself, not the list of urls, wget causes an error: No URLs found in url.

Even if the wget fails, why the command outputs just one line, not three lines as the result of md5sum? This seems to be because the head command immediately flushes the remaining lines when the piped subprocess fails.

user_85723800
Related