Backupfile deleter

Viewed 112

I have a directory full of backupfiles but need to automate the cleanup of that directory, because i run out of space. The simplest way to do it would be a powershell script.

The Backup creates 2 Backups per Day. I want to keep 2 Backups a Day for 7 Days, backups that are older than 7 days but not older than 3 months i want to keep 1 from each day and everything older than 3 months i want to keep 1 from each week. Everything older than 2 years gets deletet.

The first function i wrote deletes everything older than 2 years:

$checkPath = "C:\demo\"
$list = (Get-ChildItem -Path $checkPath | Where-Object {$_.LastWriteTime -lt (Get-Date).AddYears(-2)}).FullName
$count = $list.Length
for ($i = 0; $i -lt $count; $i += 1)
{
    Write-Verbose "[$i] $($list[$i])"
    Remove-Item -Path $list[$i] 
}

After that i create a function to delete every 2nd backup:

$checkPath = "C:\demo\"
$list = (Get-ChildItem -Path $checkPath | Where-Object {$_.LastWriteTime -lt (Get-Date).AddDays(-7)} | Where-Object {$_.LastWriteTime -lt (Get-Date).AddMonths(-3)}).FullName
$count = $list.Length
for ($i = 0; $i -lt $count; $i += 2)
{
    Write-Verbose "[$i] $($list[$i])"
    Remove-Item -Path $list[$i] 
}

Well i feel a bit lost, am i on the right path here? Does anyone may have a suggestion to do this "better"?

2 Answers

I'm not gonna go much into details here, just review the code and read the comments.

One thing to consider, on Get-ChildItem "C:\demo\" -File you might want to consider using -Filter to filter only files with the backup extension.

Note that I'm using -WhatIf flag on Remove-Item, review the script and if you consider it is doing what you want, remove this flag.

# Group all backups per day
# DateTime Format MM/dd/yyyy 12:00:00 AM
$backups = Get-ChildItem 'C:\demo\' -File |
           Group-Object {$_.CreationTime.Date} |
           Sort-Object -Descending {$_.Name -as [datetime]}
    
$date = [datetime]::Now
    
$onePerWeekGroups = foreach($group in $backups)
{
    switch($group)
    {
        {[datetime]$_.Name -ge $date.AddDays(-7)}
        {
            # Groups which Date is greater than or equal to
            # Date 7 Days ago
                
            # For this group (day), skip the first 2 backups and
            # remove the rest
            $_.Group | Select-Object -Skip 2 | Remove-Item -WhatIf
                
            break
        }
        
        {[datetime]$_.Name -lt $date.AddDays(-7) -and [datetime]$_.Name -ge $date.AddMonths(-3)}
        {
            # Groups which Date is lower than Date 7 Days ago and
            # greater than or equal to Date 3 Months ago
    
            # For this group (day), skip the first backup and
            # remove the rest
            $_.Group | Select-Object -Skip 1 | Remove-Item -WhatIf
                
            break
        }

        {[datetime]$_.Name -lt $date.AddMonths(-3) -and [datetime]$_.Name -ge $date.AddYears(-2)}
        {
            # Groups which Date is lower than Date 3 Months ago and
            # greater than or equal to Date 2 Years ago
                
            # Since this Group has more complexity than the others, 
            # will return this groups and store them in $onePerWeekGroups for later 
            return $_
        }

        Default
        {
            # Groups which Date is higher than 2 Years
                
            # For this group, remove everything
            Remove-Item -Path $_.Group.FullName -WhatIf
        }
    }
}
    
$calendar = (Get-Culture).Calendar
    
# Group all files by Year (Thanks Daniel for pointing this out)
# and by Week
$groupsPerWeek = $onePerWeekGroups.Group | Group-Object {
    $calendar.GetWeekOfYear(
        $_.CreationTime,
        [System.Globalization.CalendarWeekRule]::FirstFourDayWeek,
        [DayOfWeek]::Monday
    )}, {$_.CreationTime.Year}
    
foreach($group in $groupsPerWeek)
{
    # Skip the first file of this Week and Remove the rest
    $group.Group | Select-Object -Skip 1 | Remove-Item -WhatIf
}

Same, not going to do much detail other than what's in the code. Hopefully the comments speak loud enough

$checkPath = 'C:\temp'

$classified = switch ((Get-ChildItem -Path $checkPath -File | Group-Object { $_.LastWriteTime.Date }) ) {
    # Last 7 days
    { [datetime]::Parse(($_.Name)) -ge [datetime]::Today.AddDays(-7) } {
        $keep = $_.group | Sort-Object LastWriteTime | Select-Object -Last 2
        $keep | ForEach-Object {
            [pscustomobject]@{Path = $_.fullname; LastWriteTime = $_.LastWriteTime; Action = 'Keep'; Interval = '<= 7 days' }
        }
        $remove = $_.group | Where-Object { $_ -notin $keep }
        $remove | ForEach-Object {
            [pscustomobject]@{Path = $_.fullname; LastWriteTime = $_.LastWriteTime; Action = 'Remove'; Interval = '<= 7 days' }
        }
    }
    { [datetime]::Parse(($_.Name)) -lt [datetime]::Today.AddDays(-7) -and
        [datetime]::Parse(($_.Name)) -ge [datetime]::Today.AddMonths(-3) } {
        # Between 7 days and 3 months $_.Name -ForegroundColor Green

        $keep = $_.group | Sort-Object LastWriteTime | Select-Object -Last 1
        $keep | ForEach-Object {
            [pscustomobject]@{Path = $_.fullname; LastWriteTime = $_.LastWriteTime; Action = 'Keep'; Interval = 'Between 7 days and 3 months' }
        }
        $remove = $_.group | Where-Object { $_ -notin $keep }
        $remove | ForEach-Object {
            [pscustomobject]@{Path = $_.fullname; LastWriteTime = $_.LastWriteTime; Action = 'Remove'; Interval = 'Between 7 days and 3 months' }
        }
    }
    { [datetime]::Parse(($_.Name)) -lt [datetime]::Today.AddMonths(-3) -and
        [datetime]::Parse(($_.Name)) -ge [datetime]::Today.AddYears(-2) } {
        # Between 3 months and 2 years $_.Name -ForegroundColor Green
        # Set to unknown for now.  Group by weekOfYear later to make determination
        $_.group | ForEach-Object {
            [pscustomobject]@{Path = $_.Fullname; LastWriteTime = $_.LastWriteTime; Action = 'Unknown'; Interval = 'Between 3 months and 2 years' }
        }
    }
    Default {
        $_.group | ForEach-Object {
            # Everything after 2 years $_.Name -ForegroundColor Green
            [pscustomobject]@{Path = $_.fullname; LastWriteTime = $_.LastWriteTime; Action = 'Remove'; Interval = '> 2 years' }
        }
    }
}

# Needed to determine and group by week of year
$cultureInfo = [cultureinfo]::new('en-US')
$calendar = $cultureInfo.Calendar;
$calendarRule = $cultureInfo.DateTimeFormat.CalendarWeekRule
$firstDayOfWeek = $cultureInfo.DateTimeFormat.FirstDayOfWeek


$classified | Where-Object Interval -EQ 'Between 3 months and 2 years' |
    # Group all files between 3 months and 2 years by weekOfYear and Year and set latest one to keep, others to remove
    Group-Object { $calendar.GetWeekOfYear($_.LastWriteTime, $calendarRule, $firstDayOfWeek) }, { $_.LastWriteTime.Year } |
    ForEach-Object {
        $keep = $_.group | Sort-Object LastWriteTime | Select-Object -Last 1
        $keep | ForEach-Object { $_.Action = 'Keep' }

        $remove = $_.group | Where-Object { $_ -notin $keep }
        $remove | ForEach-Object { $_.Action = 'Remove' }
    }

$classified | Sort-Object LastWriteTime | Out-Host

$classified | Where-Object Action -EQ 'Remove' | Remove-Item -WhatIf

Output

Path                                                      LastWriteTime       Action Interval
----                                                      -------------       ------ --------
C:\temp\20210101 - 20210108.pdf                           13.01.2021 09:15:22 Keep   Between 3 months and 2 years
C:\temp\20210115 - 20210122.pdf                           26.01.2021 15:57:47 Keep   Between 3 months and 2 years
C:\temp\rm_50117.pbl                                      14.05.2021 15:26:41 Keep   Between 7 days and 3 months
C:\temp\zipped.zip                                        19.05.2021 18:35:12 Remove Between 7 days and 3 months
C:\temp\zipped2.zip                                       19.05.2021 18:40:54 Remove Between 7 days and 3 months
C:\temp\zipped3.zip                                       19.05.2021 18:46:05 Keep   Between 7 days and 3 months
C:\temp\xxx_Some Doc ID 2333 _other stuff_more stuff.junk 20.05.2021 23:32:43 Keep   Between 7 days and 3 months
C:\temp\testlksj.ps1                                      21.05.2021 22:28:03 Keep   Between 7 days and 3 months
C:\temp\skdljfsdkf.ps1                                    22.05.2021 09:04:29 Remove Between 7 days and 3 months
C:\temp\testparam.ps1                                     22.05.2021 17:08:33 Remove Between 7 days and 3 months
C:\temp\gitignorepath.txt                                 22.05.2021 21:03:06 Keep   Between 7 days and 3 months
C:\temp\lslkdfjl.ps1                                      23.05.2021 00:56:13 Remove Between 7 days and 3 months
C:\temp\test2.txt                                         23.05.2021 10:17:24 Remove Between 7 days and 3 months
C:\temp\test3.txt                                         23.05.2021 10:17:38 Remove Between 7 days and 3 months
C:\temp\test4.txt                                         23.05.2021 10:17:50 Keep   Between 7 days and 3 months
C:\temp\fromVS.txt                                        24.05.2021 22:57:44 Keep   <= 7 days
C:\temp\testlksfsdfsdj.ps1                                25.05.2021 16:55:45 Keep   <= 7 days
C:\temp\dslkfjlds.ps1                                     26.05.2021 23:35:23 Keep   <= 7 days
C:\temp\slkfjdslfj.ps1                                    27.05.2021 00:35:09 Keep   <= 7 days
C:\temp\testgit.log                                       31.05.2021 13:28:19 Keep   <= 7 days
C:\temp\out.log                                           31.05.2021 13:51:37 Keep   <= 7 days

What if: Performing the operation "Remove File" on target "C:\temp\zipped.zip".
What if: Performing the operation "Remove File" on target "C:\temp\zipped2.zip".
What if: Performing the operation "Remove File" on target "C:\temp\skdljfsdkf.ps1".
What if: Performing the operation "Remove File" on target "C:\temp\testparam.ps1".
What if: Performing the operation "Remove File" on target "C:\temp\lslkdfjl.ps1".
What if: Performing the operation "Remove File" on target "C:\temp\test2.txt".
What if: Performing the operation "Remove File" on target "C:\temp\test3.txt".
Related