Perl: close open2 handle from a background thread

Question

I am simply trying to find out how to properly use the open2 function.

See an example below. It works for a small $max, but naturally, if I write long enough to the $hIn, so eventually it will get blocked because nothing reads the data on the output continuously.

use 5.26.0;
use IPC::Open2;
my $max = 100000;
my $pid = open2(my $hOut, my $hIn, "cat") || die "failed 'cat': $!";
{
    my $cnt = 0;
    #When $max is big (e.g. 100000) so the code below will get blocked
    #when writing to $hIn
    while ($cnt<$max) {say $hIn $cnt++;}
    close($hIn) || say "can't close hIn";
}
while(<$hOut>) { print; }
close($hOut) || say "can't close hOut";
waitpid( $pid, 0 );

The only solution, that I can think about, is launching an other thread that will do the writing on the background.

With the code below I can write into the $hIn as much data as I want and read them in the main thread but the $hIn seems not to get closed. Because of that the while(<$hOut>) will never finish while waiting for more output.

use 5.26.0;
use threads;
use IPC::Open2;
my $max = 100000;
my $pid = open2(my $hOut, my $hIn, "cat") || die "failed 'cat': $!";
my $thr = threads->create(sub {
    my $cnt = 0;
    while ($cnt<$max) {say $hIn $cnt++;}
    #The close does not have any effect here (although no error is produced)
    close($hIn) || say "can't close hIn";
});
#This outputs all the data written to $hIn but never leaves the loop...
while(<$hOut> ) { print; }
close($hOut) || say "can't close hOut";
$thr->join;
waitpid( $pid, 0 );

My questions are:

Provided that my approach with threads is ok, how can I close the file handle from the background thread?
If it is not ok (actually use threads is discouraged in Perl), so can someone provide a working example of open2 that can write and read a lot of data without a risk of getting blocked waiting for the reader or writer?

EDIT: Following your suggestions here is an implementation of the code above using IPC::Run:

use 5.26.0;
use IPC::Run qw/ run /;
my $max = 1000000;
run sub {
        my $cnt = 0;
        while ($cnt<$max) {say $cnt++;}
    },
    "|", "cat",
    "|", sub {
        while(<> ) {
            print;
        }
    }
    or die "run sub | cat | sub failed: $?";

It runs without flaws, the code is very readable... I am very happy to have learned about this module. Thanks to everyone!

Yet, I consider the question to be unanswered. If it is not possible to write this functionality using open2 directly, why does that even exist and confuse people? Also not being able to close the file handle from a different thread looks like a bug to me (certainly it is - the close should at least report an error).

user_589924 · Answer

Your program stopped because the pipe to which it was writing became full.
The pipe to cat became full because cat stopped reading from it.
cat stopped because the pipe to which it was writing became full.
The pipe from cat became full because you program isn't reading from it.

So you have two programs waiting for each other to do something. This is a deadlock.

The low-level solution is to use select to monitor both ends of the pipe.

The high-level solution is to let IPC::Run or IPC::Run3 do that hard work for you.

use IPC::Run qw( run );

my $cnt_max = 100000;
my $cnt = 0;
run [ "cat" ],
   '<', sub { $cnt < $cnt_max ? $cnt++ . "
" : undef };

Perl: close open2 handle from a background thread

1 Answers