Correct way of quoting command substitution

user_1945981 0
Viewed 95

I have simple bash script which only outputs the filenames that are given to the script as positional arguments:

#!/usr/bin/env bash

for file; do
    echo "$file"
done

Say I have files with spaces (say "f 1" and "f 2"). I can call the script with a wildcard and get the expected output:

$ ./script f*
> f 1
> f 2

But if I use command substitution it doesn't work:

$ ./script $(echo f*)
> f
> 1
> f
> 2

How can I get the quoting right when my command substition outputs multiple filenames with spaces?

Edit: What I ultimatively want is to pass filenames to a script (that is slightly more elaborate than just echoing their names) in a random order, e.g. something like that:

./script $(ls f* | shuf)
command-substitution bash
4 Answers

With GNU shuf and Bash 4.3+:

readarray -d '' files < <(shuf --zero-terminated --echo f*)
./script "${files[@]}"

where the --zero-terminated can handle any filenames, and readarray also uses the null byte as the delimiter.

With older Bash where readarray doesn't support the -d option:

while IFS= read -r -d '' f; do
    files+=("$f")
done < <(shuf --zero-terminated --echo f*)
./script "${files[@]}"

In extreme cases with many files, this might run into command line length limitations; in that case,

shuf --zero-terminated --echo f*

could be replaced by

printf '%s\0' f* | shuf --zero-terminated

Hat tip to Socowi for pointing out --echo.

user_32668470

It's very difficult to get this completely correct. A simple attempt would be to use %q specifier to printf, but I believe that is a bashism. You still need to use eval, though. eg:

$ cat a.sh
#!/bin/sh

for x; do echo $((i++)): "$x"; done
$ ./a.sh *
0: a.sh
1: name
with
newlines
2: name with spaces
$ eval ./a.sh $(printf "%q " *)
0: a.sh
1: name
with
newlines
2: name with spaces
user_1407500

This feels like an XY Problem. Maybe you should explain the real problem, someone might have a much better solution.

Nonetheless, working with what you posted, I'd say read this page on why you shouldn't try to parse ls as it has relevant points; then I suggest an array.

lst=(f*)
./script "${lst[@]}"

This will still fail if you reparse it as the output of a subshell, though -

./script  $( echo "${lst[@]}" )  #  still  fails same way
./script "$( echo "${lst[@]}" )" # *still* fails same way

Thinking about how we could make it work...

user_86565520

You can use xargs:

$ ls -l
total 4
-rw-r--r-- 1 root root  0 2021-08-13 00:23 '  file  1'
-rw-r--r-- 1 root root  0 2021-08-13 00:23 '  file  2'
-rw-r--r-- 1 root root  0 2021-08-13 00:23 '  file  3'
-rw-r--r-- 1 root root  0 2021-08-13 00:23 '  file  4'
-rwxr-xr-x 1 root root 35 2021-08-13 00:25  script
$ ./script *file*
  file  1
  file  2
  file  3
  file  4
$ ls *file* | shuf | xargs -d '\n' ./script
  file  4
  file  2
  file  1
  file  3

If your xargs does not support -d:

$ ls *file* | shuf | tr '\n' '\0' | xargs -0 ./script
  file  3
  file  1
  file  4
  file  2
user_9000780
Related