How can I capture the work of an async subshell or coproc into a variable of the parent?

deinspanjer 0
Viewed 60

Summary

I have a script that gets a list of files, and needs to perform some validation on them which might take a few seconds.

In the interim, I have some user input I'd like to collect so there potentially is no visible delay while waiting for the validation to finish.

I would like to build an array variable with the list of valid files to do the final piece of work on those files.

I've tried various invocations of coproc or subshells, but I haven't managed to find any method that will work other than having the subshell write to a temp file managed by the parent and then have the parent read the contents of that temp file into the variable.

This feels to me like I need to learn how to use zsh better, so I am posing this question to the fine people monitoring this tag.

Here are two small example scripts that represent the parent as well as a proxy for the stand-alone executable that is invoked by the subshell to do the validation work.

Parent script

#!/usr/bin/env zsh

# Generate test files
echo "one" > a.txt
echo "two" >> a.txt

echo "3" > b.txt
echo "four" >> b.txt

echo "five" > c.txt
echo "6" >> c.txt

echo "seven" > d.txt
echo "eight" >> d.txt

echo "9" > e.txt
echo "10" >> e.txt

# This list is built by doing a `print -l **/*.{txt,csv}(Od)` command.
filesToValidate=($(print -l [abcde].txt))
echo "Files to validate are: $filesToValidate"

# I need to use this list later in the script, but have some other stuff that can be done
# in the interim while waiting for it to complete.
validFiles=()

# I would like to run this piece of code asynchronously
for file in $filesToValidate
do
    ./has_numerals.sh $file >/dev/null 2>&1 && validFiles+=("$file")
done
# end async code


read -q "REPLY?Have you thought about your answer long enough?"
echo .

# If the validation is done in a subshell or coproc or something,
# at this point, wait for the work to be done (if it isn't already).
wait

for file in $validFiles
do
    echo "Contents of valid file $file:"
    cat $file
done

Stand-alone validation executable example ( has_numerals.sh )

#!/usr/bin/env zsh

echo "processing $1"
sleep 1
grep -l '[0-9]' $1
zsh
1 Answers

I want to learn how to do that if it is possible.

While it is potentially possible, it is not advisable. The mktemp tool is designed for this purpose. I will show an example below.

Zsh documentation doesn't touch on the subject, but bash makes it clear it is not possible in their documentation:

Changes made to the subshell environment cannot affect the shell's execution environment.

The only way of running an asynchronous subshell and retrieving the output is to store the information in a file, write to a pipe file you later read from the parent script, or some far more elaborate method of inter-process communication.

How best to use mktemp?

Here is a version of your script reworded to use mktemp.

#!/usr/bin/env zsh

# Generate test files
echo "one" > a.txt
echo "two" >> a.txt

echo "3" > b.txt
echo "four" >> b.txt

echo "five" > c.txt
echo "6" >> c.txt

echo "seven" > d.txt
echo "eight" >> d.txt

echo "9" > e.txt
echo "10" >> e.txt

echo "11" > f.csv
echo "12" >> f.csv

# Generate the list safely from find.
filesToValidate=$(find . \( -name \*.txt -o -name \*.csv \) -print)
echo "Files to validate are: $filesToValidate"

# Make a temp directory. This should work on Linux and macOS.
TMPFILE=$(mktemp 2>/dev/null || mktemp -t 'validfiles')

( \
    echo -n "$filesToValidate" | \
    xargs -n 1 ./has_numerals.sh "$TMPFILE" \
) &
# Pass the files to xargs with only one per line (-n 1),
#     the temp file, and the file to validate.

TRAPINT() {
    # Trap for ^C
    rm -f "${TMPFILE}"
}

TRAPQUIT() {
    # Trap for ^\
    rm -f "${TMPFILE}"
}

TRAPTERM() {
    # Trap for receiving kill -TERM
    rm -f "${TMPFILE}"
}

TRAPEXIT() {
    # Trap for any exit
    rm -f "${TMPFILE}"
}

# end async code

read -q "REPLY?Have you thought about your answer long enough?"
echo .

# Wait for subshell, if required.
wait

# Loop on newline separated (@f) data from TMPFILE.
for file in "${(@f)"$(<$TMPFILE)"}"
{
    cat "$file"
}

Sample output:

% ./do.sh
Files to validate are: ./f.csv
./c.txt
./b.txt
./a.txt
./e.txt
./d.txt
Have you thought about your answer long enough?processing ./f.csv
processing ./c.txt
processing ./b.txt
processing ./a.txt
processing ./e.txt
processing ./d.txt
y.
11
12
five
6
3
four
9
10
James Risner0
Related