powershell foreach shows duplicate result

user_16801642 0
Viewed 214

I use powershell to automate extracting of selected data from a CSV file. My $target_servers also contains two the same server name but it has different data in each rows.

Here is my code:

$target_servers = Get-Content -Path  D:\Users\Tools\windows\target_prd_servers.txt
foreach($server in $target_servers) {
    Import-Csv $path\Serverlist_Template.csv | Where-Object {$_.Hostname -Like $server} | Export-Csv -Path $path/windows_prd.csv -Append -NoTypeInformation
}

After executing the above code it extracts CSV data based on a TXT file, but my problem is some of the results are duplicated.

I am expecting around 28 results but it gave me around 49.

powershell foreach
2 Answers

As commented, -Append is the culprit here and you should check if the newly added records are not already present in the output file:

# read the Hostname column of the target csv file as array to avoid duplicates
$existingHostsNames = @((Import-Csv -Path "$path/windows_prd.csv").Hostname)

$target_servers = Get-Content -Path  D:\Users\Tools\windows\target_prd_servers.txt
foreach($server in $target_servers) {
    Import-Csv "$path\Serverlist_Template.csv" |
         Where-Object {($_.Hostname -eq $server) -and ($existingHostsNames -notcontains $_.HostName)} | 
         Export-Csv -Path "$path/windows_prd.csv" -Append -NoTypeInformation
}
user_98986430

You can convert your data to array of objects and then use select -Unique, like this:

$target_servers = Get-Content -Path  D:\Users\Tools\windows\target_prd_servers.txt

$data = @()
foreach($server in $target_servers) {
    $data += Import-Csv $path\Serverlist_Template.csv| Where-Object {$_.Hostname -Like $server}
}
$data | select -Unique | Export-Csv -Path $path/windows_prd.csv -Append -NoTypeInformation

It will work only if duplicated rows have same value in every column. If not, you can pass column names to select which are important for you. For ex.:

$data | select Hostname -Unique | Export-Csv -Path $path/windows_prd.csv -Append -NoTypeInformation

It will give you list of unique hostnames.

user_122658980
Related