Combine multiple files into one by detecting source directories and files with the same names - PowerShell

user_7848350 0
Viewed 189

I am new to PowerShell.

Goal

Combine text from nested YAML files to a master YAML file.

Current setup

File structure

  • folder1/
    • (empty)
  • folder2/
    • restapi/
      • toc.yml
  • folder3/
    • Docs/
      • restapi/
        • toc.yml
  • restapi/
    • 1.json
    • 2.json
    • 3.json
    • 4.json

Contents of folder2/toc.yml

- name: JSON 1
  href: 1.json
- name: JSON 2
  href: 2.json

Contents of folder3/toc.yml

- name: JSON 3
  href: 3.json
- name: JSON 4
  href: 4.json

Desired result

File structure

  • folder1/
    • (empty)
  • folder2/
    • restapi/
      • toc.yml
  • folder3/
    • Docs/
      • restapi/
        • toc.yml
  • restapi/
    • 1.json
    • 2.json
    • 3.json
    • 4.json
    • toc.yml (generated by combining the toc.yml files in folder2/restapi and folder3/Docs/restapi)

Contents of the toc.yml file in the restapi folder at root

- name: JSON 1
  href: 1.json
- name: JSON 2
  href: 2.json
- name: JSON 3
  href: 3.json
- name: JSON 4
  href: 4.json

folder2/restapi/toc.yml and folder3/Docs/restapi/toc.yml do not need to be deleted.

Attempted code

$subfolderslist = (Get-ChildItem $PSScriptRoot -recurse | Where-Object { $_.PSIsContainer -eq $True -and $_.Name -like "restapi"} | Sort-Object)

foreach ($restapifolder in $subfolderslist) {
    $fullPath = $restapifolder.FullName
    $item = (Get-ChildItem $fullPath)
    Get-Content $fullPath/$item | Out-File -append $PSScriptRoot/restapi/toc.yml
}

This successfully produces the desired appended content of the restapi folder at root.

But it errors out with: Get-Content : Cannot find path 'C:\my-project\restapi\1.json 2.json 3.json toc.yml' because it does not exist.

I have tried adding -Exclude *.json to the $item variable but it returns an error also for both folder2 and folder3. Example: Get-Content : Cannot find path 'C:\myproject\folder3\Docs\restapi\C:\myproject\docs-multi-3\Docs\restapi\toc.yml' because it does not exist.

I have also tried excluding the restapi folder at root inside Where-Object but that fails as well.

I have tried all manner of things with changing around variables.

This is probably irrelevant but for what it's worth, this script will be executed in an Azure DevOps YAML pipeline after repos containing those folder structures are cloned (e.g., folder2 is a repo, folder3 is a repo, etc.).

Thank you for any help, including recommendations on better ways to accomplish this.

azure-pipelines powershell scripting
1 Answers

This is how I would personally do it, I created a test case so you can see all in action. I would recommend you to first create a temporary folder and cd to that temp folder then run the script to test if it's doing what you need.

Creating the test case

$yaml1 = @'
- name: JSON 1
  href: 1.json
- name: JSON 2
  href: 2.json
'@

$yaml2 = @'
- name: JSON 3
  href: 3.json
- name: JSON 4
  href: 4.json
'@

$null = New-Item restapi -ItemType Directory -Force

foreach($i in 1..3) {
    $path = [System.IO.Path]::Combine("folder$i", "Docs", "restapi")
    $ymlPath = Join-Path $path -ChildPath 'toc.yml'
    if($i -eq 2) {
        $null = New-Item $path -ItemType Directory -Force
        $yaml1 | Set-Content $ymlPath
        continue
    }
    if($i -eq 3) {
        $null = New-Item $path -ItemType Directory -Force
        $yaml2 | Set-Content $ymlPath
        continue
    }
    $null = New-Item "folder$i" -ItemType Directory -Force
}

Code Logic

$destination = Get-Item ./restapi
$filePath    = Join-Path $destination -ChildPath toc.yml
& {
    Get-ChildItem . -Recurse -Filter *.yml | Sort-Object {
        # Sort the Yaml files starting from the first numbered folder
        # `$_.Directory.Parent.Parent` would be the numbered folders:
        #  folder1, folder2, etc..
        [int][regex]::Match($_.Directory.Parent.Parent, '\d+').Value
    } | Get-Content -Raw
} | Set-Content $filePath

# Test if it worked
Get-Content $filePath
user_153395440
Related