How can I move an array of files dynamically with Powershell and RoboCopy to individual subfolders of a fixed size?

Viewed 31

I am creating a script that splits a target folder's files into subfolders of n length, where n is a number specified dynamically.

So basically, if Folder A has 9000 files, and I limit the number of files to 1000 per folder, the script would create nine sub-directories inside of Folder A with 1000 files each.

Here is working code:

param (
    [Parameter(Mandatory,Position=0)]
    [String]
    $FileList,

    [Parameter(Mandatory=$false,ValueFromPipelineByPropertyName)]
    [Int32]
    $NumFilesPerFolder = 1000,

    [Parameter(Mandatory=$false,ValueFromPipelineByPropertyName)]
    [Int32]
    $FolderNumberPadding = 2
)

$Folders = Get-Content $FileList
Set-Location -LiteralPath ([IO.Path]::GetTempPath())

function Move-Files {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory,Position=0)]
        [System.Collections.ArrayList]
        $List,

        [Parameter(Mandatory)]
        [Int32]
        $Index
    )

    $BaseFolder      = [System.IO.Path]::GetDirectoryName($List[0])
    $DestFolderName  = $Index.ToString().PadLeft($FolderNumberPadding, '0')
    $DestFolder       = New-Item -Path (Join-Path $BaseFolder $DestFolderName) -Type Directory -Force

    Move-Item $List -Destination $DestFolder -Force

}


foreach ($Folder in $Folders) {

    $Files = Get-ChildItem -LiteralPath $Folder -File -Force

    $filesidx = 1
    $totalidx = $null
    $groupidx = 0

    $FilesToMove = [System.Collections.ArrayList]@()

    foreach ($File in $Files) {
        if($null -eq $totalidx){
            $totalidx = $Files.Length
        }
        if($filesidx -eq 1){
            $groupidx++
        }

        $FilesToMove.Add($File)

        if($filesidx -eq $NumFilesPerFolder){
            Move-Files -List $FilesToMove -Index $groupidx
            $FilesToMove.Clear()
            $filesidx = 1
        }elseif($totalidx -eq 1){
            Move-Files -List $FilesToMove -Index $groupidx
            $FilesToMove.Clear()
            break
        }else{
            $filesidx++
        }

        $totalidx--
    }
}

Remove-Item $FileList -Force

$app = New-Object -ComObject Shell.Application
$appwin = $app.Windows()

foreach ($window in $appwin) {
    if($window.Name -eq "File Explorer"){
        $window.Refresh()
    }
}


Invoke-VBMessageBox "Operation Complete" -Title "Operation Complete" -Icon Information -BoxType OKOnly

This code runs reasonably well, but it heavily bottlenecks when actually moving the files with Move-Item. I'd like to try and use RoboCopy here, but I am perplexed as to how I can implement it.

What I'm having trouble with is that the items I need to move are stored in a list (see the Move-Files function), and every item that needs to be moved are all in the same sub-directory. So I can't just do RoboCopy.exe C:\Source C:\Destination /mov.

How can I integrate RoboCopy here to accomplish my goal? I really need multi-threaded performance as this function will be responsible for moving thousands of files around in production on a frequent basis.

Any help would be greatly appreciated - please let me know if I can provide more information to further clarify my objective.

Thanks for any help at all!

0 Answers
Related