Using "expanding characters" in a variable in a bash script

Question

I apologize beforehand for this question, which is probably both ill formulated and answered a thousand times over. I get the feeling that my inability to find an answer is that I don't quite know how to ask the question.

I'm writing a script that traverses folders in a bunch of mounted external hard drives, like so:

for g in /Volumes/compartment-?/{Private/Daniel,Daniel}/Projects/*/*

It then proceeds to perform long-running tasks on each of the directories found there. Because these operations are io-intensive rather than cpu-intensive, I thought I'd add the option to provide which "compartment" I want to work in, so that I can parallelize the workloads.

But, doing

cmp="?"
[[ ! "$1" = "" ]] && cmp="$1"

And then, for g in /Volumes/compartment-$cmp/{Private/Daniel,Daniel}/Projects/*/*

Doesn't work - the question mark that should expand to all compartments instead becomes literal, so I get an error that "compartment-?" doesn't exist, which is of course true.

How do I create a variable with a value that "expands," like dir="./*" working with ls $dir?

EDIT: Thanks to @dan for the answer. I was brought up to be courteous and thank people, so I did thank him for it in a comment on his question, but that comment has been removed, and I'm anxious that repeating it might be some kind of infraction here. I ended up simply escaping my question mark glob character, i.e. \?, since for this script I only need to either search all drives or one particular drive. But I'll keep the answer handy for the next time I write a script where I'd like to support more advanced arguments.

user_13919668 · Answer

Brace expansion occurs before variable expansion. Pathname/glob expansion (eg ?, *) occurs last. Therefore you can't use the glob character ? in a variable, and in a brace expansion.

You can use a glob expression in an unquoted variable, without brace expansion. Eg. q=\?; echo compartment-$q is equivalent to echo compartment-?.

To solve your problem, you could define an array based on the input argument:

if [[ $1 ]]; then
    [[ -d /Volumes/compartment-$1 ]] || exit 1
    files=("/Volumes/compartment-$1"/{Private/Daniel,Daniel}/Projects/*/*)
else
    files=(/Volumes/compartment-?/{Private/Daniel,Daniel}/Projects/*/*)
fi

# then iterate the list:
for i in "${files[@]}"; do
...

Another option is a nested loop. The path expression in the outer loop doesn't use brace expansion, so (unlike the first example) it can expand a glob in $1 (or default to ? if $1 is empty):

for i in /Volumes/compartments-${1:-?}; do
    [[ -d $i ]] &&
    for j in {Private/Daniel,Daniel}/Projects/*/*; do
        [[ -e $j ]] || continue
        ...

Note that the second example expands a glob expression passed in $1 (eg. ./script '[1-9]'). The first example does not.
Remember that pathname expansion has the property of expanding only to existing files, or literally. shopt -s nullglob guarantees expansion only to existing files (or nothing).
You should either use nullglob, or check that each file or directory exists, like in the examples above.
Using $1 unquoted also subjects it to word splitting on whitespace. You can set IFS= (empty) to avoid this.

Using "expanding characters" in a variable in a bash script

1 Answers