If you make your function an advanced function, by way of [CmdletBinding()], like this:
function Foo {
[CmdletBinding()]
param()
for($i = 0; $i -lt 10; $i++) {
Write-Host Inner $i; $i;
}
}
Then you gain access to the automatic parameter -OutVariable, and now you can do this:
Foo -OutVariable foo | Select-Object -First 5
$foo
Of course, in your example, you could just as easily do:
$foo = Foo | Select-Object -First 5
So I'm not certain exactly what you want, but I do suspect that -OutVariable fits the bill better, because it is populated as the pipeline goes.
So for example:
function Foo {
[CmdletBinding()]
param()
for($i = 0; $i -lt 10; $i++) {
Write-Host Inner $i; $i;
sleep -s 2
}
}
(now the function sleeps 2 seconds on every iteration)
If you then CTRLC in the middle of it, $foo will contain what was produced so far.
With further comments, I think I see what you're after now: a return value from the function that can be enumerated later. This is an Enumerator.
There are two complications to returning one from your function:
- PowerShell tends to automatically "unroll" (process) these on return, but you can probably get around this with the classic trick of returning a single element array, where the element is the enumerator.
- Creating the enumerator without pre-processing all the items though is going to be dependent on the specifics of it.
To create an enumerator you'll need to create a type (a class) that inherits from IEnumerator. You can make this class do whatever you want so that's where the complication of the implementation will go.
Your function then will really just serve as a PowerShell interface to instantiating and returning such a thing.
class MyEnumerator : System.Collections.IEnumerator
{
hidden [int] $index;
hidden [int] $max;
hidden [int] $start;
hidden [int] $item;
MyEnumerator([int]$max, [int]$start=0) {
$this.index = -1
$this.start = $start
$this.max = $max
}
[bool] MoveNext() {
++$this.index
$this.item = $this.index + $this.start
# demonstration
if ($this.item -gt 7) {
throw
}
return ($this.item -le $this.max)
}
[void] Reset() {
$this.index = -1
}
[object] get_Current() {
return $this.item
}
}
function Foo ($start, $max) {
,[MyEnumerator]::new($max, $start)
}
So here in this example, I've created this enumerator class, and you can give it any start and max integer value for it to enumerate through. But I threw in a little catch in that it's hardcoded to throw an exception if the item value is greater than 7.
The function creates the enumerator and returns it (notice the unary comma for making a single element array).
So you can run this like so:
$foo = Foo 2 10
# no exception!
$foo | Select -First 5
# no exception!
This on the other hand:
$foo = Foo 2 10
$foo
# will throw
(you've attempted to enumerate through the whole thing, implicitly)
One thing you need to be aware of is that $foo is the enumerator object, and keeps state, and now you need to be responsible for that if you're trying to "re-use" it.
Example:
$foo = Foo 2 10
$foo | Select -First 5
$foo.Current
# 6
$foo | Select -First 5
# 7
# exception!
You can use $foo.Reset() to make it go back to the beginning.
Another alternative: returning a function (or just a script block) from a function.
This one won't behave as naturally as the enumerator, but it'll be easier to implement.
function Foo {
{
for($i = 0; $i -lt 10; $i++) { Write-Host Inner $i; $i; }
}
}
$foo = Foo ; & $foo | Select-Object -First 5
For this, since the return value is a script block, you'll have to execute it before piping it along.
I imagine what you were really looking for was the enumerator, but you can see that that will require some heavier lifting to implement.
Ok so now seeing your more realistic examples from your edit:
Get-ChildItem -Directory -Recurse |
Where-Object { Test-Path (Join-Path $_.FullName ".git") } |
Select-Object -ExpandProperty FullName |
Get-ChildItem -File -Recurse -Filter *.cs |
Get-Content |
Select-String "public struct" |
Select-Object -First 1;
vs
$gitRepositories = Get-ChildItem -Directory -Recurse |
Where-Object { Test-Path (Join-Path $_.FullName ".git") };
$csharpFiles = $gitRepositories |
Select-Object -ExpandProperty FullName |
Get-ChildItem -File -Recurse -Filter *.cs
$structNames = $csharpFiles |
Get-Content |
Select-String "public struct" |
Select-Object -First 1;
I actually still don't see the point of the intermediate variables here. You're kind of fighting the tool. I don't really think they add much to readability, and as you've already seen, they harm performance.
That said, for finding files and directories, you can almost do exactly what you want.
Here's a replacement for the first part, $gitRepositories:
$initialPath = Get-Location
$top = [System.IO.DirectoryInfo]::new($initialPath)
$gitRepositories = $top.EnumerateDirectories('.git', [System.IO.SearchOption]::AllDirectories)
In this example, $gitRepositories will be an enumerable, so it will not enumerate the directories until forced to. You could use it with foreach or just try to display it and at that moment it will go through the tree. Those things in PowerShell know how to deal with enumerables already; that is, they call .GetEnumerator() automatically and use the resulting enumerator.
You can do that yourself too, $e = $gitRepositories.GetEnumerator() and then making calls to $e.MoveNext() and $e.Current, etc.
But the part where this will start to break down is your second one:
$csharpFiles = $gitRepositories |
Select-Object -ExpandProperty FullName |
Get-ChildItem -File -Recurse -Filter *.cs
The issue here is that you need to enumerate the first completely, in order to get the next enumerable, so as soon as you want this intermediate one, you will lose the "lazy" aspect for the first completely.
The other issue is that there isn't just one directory on which to call .EnumerateFiles(); $gitRepositories can contain many directories and you need to enumerate the files of each one, so you can't even "take the hit" of enumerating $gitRepositories and end up with a single enumerable as the result... at least not without creating some kind of enumerable class of your own that accepts a list of enumerables and then enumerates them all in sequence, providing a "virtual" single enumerable.
All that being said while this approach can't give you exactly what you are looking for, in general the [DirectoryInfo] object's .Enumerate* methods are quite useful, and very performant, and the resultant [DirecotryInfo] and/or [FileInfo] obejcts are also chock full of useful properties and methods, providing nearly all the functionality of the objects returned from the PowerShell-native cmdlets, so you should consider using these in general!
The last option I have for you is kind of nuts, and it's not going to make any of your code easier to read or understand, that's for sure.
Unlike your initial sample functions, the commands you're trying to run in your more real-world example are true cmdlets that support begin/process/end. The "process" part (what would correspond to a Process {} block in an advanced function) does sort of act an enumerator in a pipeline.
You can actually create some objects that let you call commands with fine-grained control over running those pieces. This is how "proxy functions" work, which is also how implicit remoting works.
You can have PowerShell generate such a proxy command for you, with [System.Management.Automation.ProxyCommand]::Create().
$gci = Get-Command Get-ChildItem
[System.Management.Automation.ProxyCommand]::Create($gci)
The result will be this big blob of stuff:
[CmdletBinding(DefaultParameterSetName='Items', SupportsTransactions=$true, HelpUri='https://go.microsoft.com/fwlink/?LinkID=113308')]
param(
[Parameter(ParameterSetName='Items', Position=0, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string[]]
${Path},
[Parameter(ParameterSetName='LiteralItems', Mandatory=$true, ValueFromPipelineByPropertyName=$true)]
[Alias('PSPath')]
[string[]]
${LiteralPath},
[Parameter(Position=1)]
[string]
${Filter},
[string[]]
${Include},
[string[]]
${Exclude},
[Alias('s')]
[switch]
${Recurse},
[uint32]
${Depth},
[switch]
${Force},
[switch]
${Name})
dynamicparam
{
try {
$targetCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Management\Get-ChildItem', [System.Management.Automation.CommandTypes]::Cmdlet, $PSBoundParameters)
$dynamicParams = @($targetCmd.Parameters.GetEnumerator() | Microsoft.PowerShell.Core\Where-Object { $_.Value.IsDynamic })
if ($dynamicParams.Length -gt 0)
{
$paramDictionary = [Management.Automation.RuntimeDefinedParameterDictionary]::new()
foreach ($param in $dynamicParams)
{
$param = $param.Value
if(-not $MyInvocation.MyCommand.Parameters.ContainsKey($param.Name))
{
$dynParam = [Management.Automation.RuntimeDefinedParameter]::new($param.Name, $param.ParameterType, $param.Attributes)
$paramDictionary.Add($param.Name, $dynParam)
}
}
return $paramDictionary
}
} catch {
throw
}
}
begin
{
try {
$outBuffer = $null
if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
{
$PSBoundParameters['OutBuffer'] = 1
}
$wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Management\Get-ChildItem', [System.Management.Automation.CommandTypes]::Cmdlet)
$scriptCmd = {& $wrappedCmd @PSBoundParameters }
$steppablePipeline = $scriptCmd.GetSteppablePipeline($myInvocation.CommandOrigin)
$steppablePipeline.Begin($PSCmdlet)
} catch {
throw
}
}
process
{
try {
$steppablePipeline.Process($_)
} catch {
throw
}
}
end
{
try {
$steppablePipeline.End()
} catch {
throw
}
}
<#
.ForwardHelpTargetName Microsoft.PowerShell.Management\Get-ChildItem
.ForwardHelpCategory Cmdlet
#>
The important bits for you are all around creating a "steppable pipeline" around the command, which would let you call .Process() yourself; that is to invoke individual iterations of the process pipeline.
In theory you could cobble together some sort of wrapper function, specialized enumerable class, etc. that could result in single "variables" that let you lazily enumerate through pipelines.
I consider this more of a thought exercise than something practical though.
