Configure depth overflow value - Start-Job

Question

I have a recursive function that is executed around 750~ times - iterating over XML files and processing. The code is running using Start-Job

Example below:

$job = Start-Job -ScriptBlock {

    function Test-Function {

        Param 
        (
            $count
        )
        Write-Host "Count is: $count"

        $count++
        Test-Function -count $count
    }
    Test-Function -count 1

}

Output:

$job | Receive-Job
Count is: 224
Count is: 225
Count is: 226
Count is: 227
The script failed due to call depth overflow.

The depth overflow occurs at 227 consistently on my machine. If I remove Start-Job, I can reach 750~ (and further). I am using jobs for batch processing.

Is there a way to configure the depth overflow value when using Start-Job?

Is this a limitation of PowerShell Jobs?

user_934946 · Answer

I can't answer about the specifics of call depth overflow limitations in PS 5.1 / 7.2 but you could do your recursion based-off a Queue within the job.

So instead of doing the recursion within the function, you do it from outside (still within the job though).

Here's what this look like.

$job = Start-Job -ScriptBlock {
$Queue = [System.Collections.Queue]::new()

    function Test-Function {

        Param 
        (
            $count
        )
        Write-Host "Count is: $count"

        $count++
        # Next item to process.
        $Queue.Enqueue($Count)
    }
    
    # Call the function once
    Test-Function -count 1
    # Process the queue
    while ($Queue.Count -gt 0) {
        Test-Function -count $Queue.Dequeue()
    }
}

Reference:

.net Queue class

user_15339544 · Answer

Not an answer that would solve the problem but rather an informative one. You could use an instance of the PowerShell SDK's [powershell] class instead of Start-Job, which can handle more levels of recursion (by a big amount) in case it helps, here are my results:

Technique	Iterations	PowerShell Version	Operating System
`Start-Job`	226~	5.1	Windows 10
`Start-Job`	2008~	7.2.1	Linux
PowerShell Instance	4932~	5.1	Windows 10
PowerShell Instance	11193~	7.2.1	Linux

Code to reproduce

$instance = [powershell]::Create().AddScript({
    function Test-Function {
        Param($count)
        Write-Host "Count is: $count"
        $count++
        Test-Function -count $count
    }
    Test-Function -count 1
})
$handle = $instance.BeginInvoke()
do {
    $done = $handle.AsyncWaitHandle.WaitOne(500)
} until($done)
$instance.Streams.Information[-1]
$instance.Dispose()

Configure depth overflow value - Start-Job

2 Answers