PowerShell Retry Loop for fail handling and retry

There are many ways you can do this, here is one example using recursion. I have stripped your function out of it's original code just to show you how the logic works.

1. Add a -Retries parameter to the function, leaving it's default value to 2 for testing. This value can be changed later and also when calling the function -Retries X can be used if more retries are needed later on the road.
2. If the try block fails, subtract 1 from the counter in the catch block.
3. If $Retries is not yet 0 (-not $PSBoundParameters.Retries), make the function call itself passing the same arguments.
4. If $Retries has reached 0, send the email and end the function.

function Download-FilesFromSharePointSiteLibrary {
[cmdletbinding()]
param (
    [Parameter(Mandatory=$true)] [string] $SiteURL,
    [Parameter(Mandatory=$true)] [string] $LibraryName,
    [Parameter(Mandatory=$true)] [string] $TargetFolder,
    [Parameter()] [int] $Retries = 2 # Retries 2 times by Default
)

    Try {
        # Original code goes here
        throw # Remove this after Testing
    }
    Catch {
        Write-Warning "Script Failed retrying..."
        Start-Sleep -Seconds 2
        # Add new subtracted Value to `$PSBoundParameters`
        $PSBoundParameters['Retries'] = (--$Retries)
        
        if(-not $PSBoundParameters.Retries) {
            Write-Warning "We are out of retries, sending Email"
            # If we're out of retries
            # Send Mail logic goes here
            return
        }
        Download-FilesFromSharePointSiteLibrary @PSBoundParameters
    }
}

Download-FilesFromSharePointSiteLibrary -SiteURL hello -LibraryName world -TargetFolder something

Without modifying your code too much, you can increment the amount of times it's ran before calling on itself again using a $global: variable.

# Description: Download All Files from SharePoint Site Library, then Delete files from SharePoint.
# --------------------------------------------------

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

Function Download-FilesFromSharePointSiteLibrary
{
    param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $LibraryName,
        [Parameter(Mandatory=$true)] [string] $TargetFolder
    )
    Begin 
    {
        if (-not $global:retriedTimes) {
            $global:retriedTimes = 0
        }
        $removeRetry = { Remove-Variable -Name "retriedTimes" -Scope "Global" }
    }
    Process
    {
        Try {
            # Credentials to Connect to SharePoint
            $Username = "Test.User@email.com"
            $Password = Get-Content "C:\SharePoint\SharePoint-Credentials.txt" | ConvertTo-SecureString
            $Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Username, $Password)

            # Login to SharePoint Client
            $Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
            $Ctx.Credentials = $Credentials

            # Get Files from SharePoint Site Library
            $List = $Ctx.Web.Lists.GetByTitle($LibraryName)
            $Ctx.Load($List)
            $Ctx.ExecuteQuery()

            # Get Files from SharePoint Site Library Folder
            $Folder = $List.RootFolder
            $FilesColl = $Folder.Files
            $Ctx.Load($FilesColl)
            $Ctx.ExecuteQuery()

            Foreach($File in $FilesColl)
            {
                $TargetFile = $TargetFolder+$File.Name
                # Download File from SharePoint Library Folder
                $FileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($Ctx,$File.ServerRelativeURL)
                $WriteStream = [System.IO.File]::Open($TargetFile,[System.IO.FileMode]::Create)
                $FileInfo.Stream.CopyTo($WriteStream)
                $WriteStream.Close()
                # Delete File from SharePoint Library Folder
                $File.DeleteObject()
                $Ctx.ExecuteQuery()
            }
            # Success - End Program
            Write-host -f Green "Success!" $_.Exception.Message
            & $removeRetry
        }
        Catch {
            $global:retriedTimes += 1
            if ($global:retriedTimes -ge 2) {
                Send-MailMessage -From "no-reply@email.com" -To "Test.User@email.com" -Subject "Failure: SharePoint Connector" -Body "Failure occurred with SharePoint Connector on Server.`n`n$_" -Priority "High" -SMTPServer "smtp.email.com" -Port "25"
                & $removeRetry
                Break
            }
            # Failure - Sleep for 5 Minutes
            Write-host -f Red "Failure!" $_.Exception.Message
            Start-Sleep -Seconds 300
            Download-FilesFromSharePointSiteLibrary @PSBoundParameters
        }
    }
}

# SharePoint Site Parameters and Target Directory
$SiteURL="https://site.sharepoint.com/sites/Test"
$LibraryName="Library"
$TargetFolder="C:\SharePoint\Files\"

# Run Program
Download-FilesFromSharePointSiteLibrary -SiteURL $SiteURL -LibraryName $LibraryName -TargetFolder $TargetFolder

• Added a Begin, and Process block to implement the global variable of amount of retries.