Process.StandardOutput.Readline() is hanging when there is no output

• StreamReader.ReadLine() is blocking by design.

• There is an asynchronous alternative, .ReadLineAsync(), which returns a Task<string> instance that you can poll for completion, via its .IsCompleted property, without blocking your foreground thread (polling is your only option in PowerShell, given that it has no language feature analogous to C#'s await).

Here's a simplified example that focuses on asynchronous reading from a StreamReader instance that happens to be a file, to which new lines are added only periodically; use Ctrl-C to abort.

I would expect the code to work the same if you adapt it to your stdout-reading System.Diagnostics.Process code.

# Create a sample input file.
$n=3
1..$n > tmp.txt

# Open the file for reading, and convert it to a System.IO.StreamReader instance.
[IO.StreamReader] $reader = 
  [IO.File]::Open("$pwd/tmp.txt", 'Open', 'Read', 'ReadWrite')

try {
  $task = $reader.ReadLineAsync() # Start waiting for the first line.
  while ($true) { # Loop indefinitely to wait for new lines.
    if ($task.IsCompleted) {  # A new line has been received.
      $task.Result # Output
      # Start waiting for the next line.
      $task.Dispose(); $task = $reader.ReadLineAsync(); 
    }
    else { # No new line available yet, do other things.
      Write-Host '.' -NoNewline
      Start-Sleep 1
    }
    # Append a new line to the sample file every once in a while.
    if (++$n % 10 -eq 0) { $n >> tmp.txt }
  }
}
finally {
  $reader.Dispose()
}

The StreamReader from standard output is waiting for IO - the stream from StandardOutput can't get opened for reading until it gets more data or the Process is closed. There's a similar issue with using StreamReader on TCP streams, where you get to enjoy waiting for network traffic.

The usual way around it is to let a different task wait on it via async reads like ReadLineAsync(). This doesn't help your while loop, because it still sits and waits the same amount before it can progress to read the output's StreamReader.

If packer.exe's output behaves well, you can try using a powershell job's output property instead of [process]? I tested with ping that gives the same waiting-for-IO lockup behavior:

# Locks up for full ping timeout example: 
$pStartProps = @{ FileName  = 'ping.exe' ; Arguments = '10.bad.ip.0' ... }
$p = [System.Diagnostics.Process]::Start($pStartInfo)

# Locks when reading
$output = $p.StandardOutput.ReadLine()
while ( $output -ne $null ) {
  $output = $p.StandardOutput.ReadLine(); ## read the output
  # Do stuff
}

#####

# Does not lock when using Job output instead:
$job = Start-Job -ScriptBlock { ping.exe 10.bad.ip.0 }

While ($job.State -ne 'Completed') {
  $job.ChildJobs[0].Output  ## read the output
  # Do stuff
}

Using StandardOutput.ReadLine() as above, you set the Output redirection stream to use Synchronous read mode. This means that when you call $p.StandardOutput.ReadLine() it behaves like a normal function call.

You call it, and wait for the return value. The return value is either a complete line or $null if the end of stream is reached. This means that as the running program fills the System.Diagnostics.Process output stream buffer with output, you continue to be a good synchronous PowerShell boy and wait for the function call to return. Once the System.Diagnostics.Process detects a newline in the stream, the function call $p.StandardOutput.ReadLine() returns with your data, a full complete line.

This is why it seems like it is blocking. It has to wait till it has a full line before the function can return.

The other return value, $null, makes more sense knowing that a StreamReader object is used for many different things that could have different endings for the stream. If you were using a StreamReader to read a text file, the end of the stream/file is when you hit the EOF marker. If you were reading in an array of characters, it could be the \0 string terminator. $null in this case is a universal end of stream.

For a System.Diagnostics.Process object, it's definition of "the end of a stream" is when the process completes, not "I have no output at this moment".

In order to remove this blocking you have to perform your reading in an Asynchronous manner a.k.a. ReadLineAsync().

Not wanting to step on @mklement0 toes, as he, correctly, solves your problem by using ReadLineAsync() to make it Asynchronous, and by polling to keep the checking/validation script inside the same scope. But an alternative method to read data asynchronously is to use Process.BeginOutputReadLine() and the OutputDataReceived Event to Asynchronously fire off a Script Block when you have more data.

# Script to run when you have another line of data
$NewOutputSB = {
    param([object]$sender, [System.Diagnostics.DataReceivedEventArgs]$e)
    Write-Host "  #  " $e.Data
}

$pStartProps = @{
    FileName               = ( Get-Command -CommandType Application ping.exe ).Source
    Arguments              = "8.8.8.8"
    UseShellExecute        = $false
    RedirectStandardOutput = $true
    RedirectStandardError  = $false
    LoadUserProfile        = $true
}
$pStartInfo = New-Object System.Diagnostics.ProcessStartInfo -Property $pStartProps

$p = New-Object System.Diagnostics.Process

# Register the Event Handler
$EventSub = Register-ObjectEvent $p -EventName OutputDataReceived -Action $NewOutputSB

$p.StartInfo = $pStartInfo

$p.Start() | Out-Null

# Begin asynchronous reading on the Output stream
$p.BeginOutputReadLine()

# Do other work while we wait for it to finish running...
while ( !$p.HasExited ) {
    # Do stuff
    Start-Sleep -Milliseconds 250
    Write-Host "--- Do stuff"
}

# Cleanup Event Handler
Unregister-Event -SubscriptionId $EventSub.Id

After running this code, the Output shows the stream running in Asynchronous mode:

PS C:\>
  #   
  #   Pinging 8.8.8.8 with 32 bytes of data:
  #   Reply from 8.8.8.8: bytes=32 time=28ms TTL=119
--- Do stuff
--- Do stuff
--- Do stuff
  #   Reply from 8.8.8.8: bytes=32 time=28ms TTL=119
--- Do stuff
--- Do stuff
--- Do stuff
--- Do stuff
  #   Reply from 8.8.8.8: bytes=32 time=31ms TTL=119
--- Do stuff
--- Do stuff
--- Do stuff
--- Do stuff
  #   Reply from 8.8.8.8: bytes=32 time=28ms TTL=119
  #
  #   Ping statistics for 8.8.8.8:
  #       Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
  #   Approximate round trip times in milli-seconds:
  #       Minimum = 28ms, Maximum = 31ms, Average = 28ms
  #
--- Do stuff