Possible to slow down a PowerShell Recursive Search?

user_7463503 0
Viewed 244

This is going to be a weird one for ya.

I need to rate limit or slow down a powershell one liner that I am using to search a local file system, as well as network shares in an enterprise environment. I would like to slow down the search to minimize the possibility of network impact.

The script searches files (mostly document or text files) for certain keywords and phrases. Getting this done quickly is not an issue as I am not on a time crunch, safety is key.

Here is the one liner:

Get-ChildItem -path C:\ -recurse -Filter *.txt -ErrorAction Continue | Select-String -Pattern "xxxx" | select filename, Linenumber, Line, Path | Format-Table
powershell
2 Answers

With your code added then, I think I was on the right track. With your original code (formatted a bit):

Get-ChildItem -path C:\ -recurse -Filter *.txt -ErrorAction Continue | 
    ForEach-Item -Process {
        Start-Sleep -Seconds 1
        $_
    } |
    Select-String -Pattern "xxxx" | 
    select filename, Linenumber, Line, Path | 
    Format-Table

Want to make it look more idiomatic? Write your own function that accepts pipeline input and delays execution. I might even use a filter which is a shorthand way of writing a pipeline-aware function:

filter Delay-Object ([int]$Milliseconds) {
    Start-Sleep -Milliseconds $Milliseconds
    $_
}

Get-ChildItem -path C:\ -recurse -Filter *.txt -ErrorAction Continue | 
    Delay-Object -Milliseconds 1000 |
    Select-String -Pattern "xxxx" | 
    select filename, Linenumber, Line, Path | 
    Format-Table

Without your code this is pure speculation but let's say you're doing something like this:

Get-ChildItem \\my\share\*.* | ForEach-Object {
    # do your search here
}

You can just introduce a delay right into your iteration:

Get-ChildItem \\my\share\*.* | ForEach-Object {
    Start-Sleep -Seconds 1
    # do your search here
}

If you're not using your own script block, let's say you're using Select-String:

Get-ChildItem \\my\share\*.* | Select-String findme

Then the solution is the same: insert a ForEach-Object!

Get-ChildItem \\my\share\*.* | 
    ForEach-Object {
        Start-Sleep -Seconds 1
        $_  # have to return the original object back to the pipeline
    } |
    Select-String findme

You might be thinking that Get-ChildItem is going to find all the files first and pass them all along and that even that will be too much stress on the network drives, but the pipeline doesn't work that way. *

Each item found is going to be passed to the next command in the pipeline one by one, so your delays will happen between each item. Therefore, you can basically insert a delay between any pipeline commands.

* some pipeline commands like Sort-Object need to collect all of the items and then pass them all out at once; from the POV of the next commands it still looks the same, but it will change how/where you need to put delays.

user_39050790

Invoke-command has a -throttle parameter which might help. It limits the number of threads so might help limit the throughput.

user_107956900
Related