Dispose of a StreamWriter without declaring a variable in one line

user_4697349 0
Viewed 106

The following Powershell command fails to copy the entire file; a few characters are always missing from the end.

[System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8).Write([System.IO.StreamReader]::new('C:\Temp\a.csv', [System.Text.Encoding]::GetEncoding('iso-8859-1')).ReadToEnd())

I suspect it's because the writer doesn't flush the last bits because this does copy the entire file:

$X = [System.IO.StreamReader]::new('C:\Temp\a.csv', [System.Text.Encoding]::GetEncoding('iso-8859-1'))
$Y = [System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)
$Y.Write($X.ReadAll())
$X.Dispose()
$Y.Dispose()

Is it possible to dispose of (and flush) the reader & writer without having created variables to reference them?

EDIT: I tried this one-liner using streamreader/writer hoping the reader's read buffer would directly transfer to the writer's write buffer rather than waiting for the reader to read the entire file into memory and then write. What technique might achieve that?

I personally find code that does not declare a single-use object to often be cleaner / more succinct, but my focus is on understanding whether/how objects dispose of themselves, not the style.

There's no need to eschew variables or write on one line, but this behaviour isn't what I expected. In VBA one can copy a file like so and trust it will dispose of itself properly without having to declare a variable and explicitly flush (I think).

Sub Cpy()
With New Scripting.FileSystemObject
    .CreateTextFile("c:\Temp\Out.txt").Write .OpenTextFile("C:\Temp\In.txt", ForReading).ReadAll
End With
End Sub

One can achieve similar behaviour in a custom VBA class by writing appropriate 'clean-up' code in a Class_Terminate() procedure. I assumed the Streamwriter would similarly flush data upon termination via garbage collection once the line executes and there's no longer a variable associated with it.

I also noticed that the file remains locked and I cannot delete it until I close the powershell session. Is there a way to flush contents and release the file without having declared a variable to work with?

resource-cleanup idisposable powershell .net
2 Answers

Just to show you that this is possible, and easier to do, using the static methods of System.IO.File, WriteAllText() and ReadAllText().

The following queries the https://loripsum.net/ API to get random paragraphs and writes to a file using the iso-8859-1 encoding. Then reads that files and writes a copy using the same encoding and lastly compares both file hashes. As you can see reading and writing is all done as a one-liner.

The using statements can be removed but you would need to use the Full Type Names.

Set location to a temporary folder for testing.

using namespace System.IO
using namespace System.Text

$fileRead = [Path]::Combine($pwd.Path, 'test.txt')
$fileWrite = [Path]::Combine($pwd.Path, 'test-copy.txt')

$loremIpsum = Invoke-RestMethod 'https://loripsum.net/api/5/short/headers/plaintext'
[File]::WriteAllText($fileWrite, $loremIpsum, [Encoding]::GetEncoding('iso-8859-1'))

[File]::WriteAllText(
    $fileWrite,
    [File]::ReadAllText($fileRead, [Encoding]::GetEncoding('iso-8859-1')),
    [Encoding]::GetEncoding('iso-8859-1')
)

(Get-FileHash $fileRead).Hash -eq
(Get-FileHash $fileWrite).Hash # => Should be True
user_153395440

  • For the specific use case given, Santiago Squarzon's helpful answer is indeed the best solution: using the static methods of the static System.IO.File class obviates the need for instances representing files that require calling a .Close() method or explicit disposing of.

    • To read lazily and therefore support overlapping reading and writing, line by line, you can use the static [System.IO.File]::ReadLines() and [System.IO.File]::WriteAllLines() methods, but note that this approach (a) invariably uses platform-native [Environment]::NewLine-format newlines in the output file, irrespective of what newline format the input file uses, and (b) invariably adds a trailing newline in this format, even if the input file had no trailing newline.

    • Overcoming these limitations would require use of a lower-level, raw-byte API, System.IO.FileStream - which again requires explicit disposal (see bottom section).

  • Given that your approach reads the entire input file into memory first and then writes, you could even make do with PowerShell cmdlets, assuming you're running PowerShell (Core) 7+, which writes BOM-less UTF-8 files by default, and whose -Encoding parameter accepts any supported encoding, such as ISO-8859-1 in your case:

    # PowerShell (Core) 7+ only
Get-Content -Raw -Encoding iso-8859-1 C:\TEMP\a.csv |
  Set-Content -NoNewLine C:\TEMP\b.csv

As for your general question:

As of PowerShell (Core) 7.2.1:

  • PowerShell has no construct equivalent to C#'s using statement that allows automatic disposing of objects whose type implements the System.IDisposable interface (which, in the case of file I/O APIs, implicitly closes the files).

    • GitHub issue #9886 discusses adding such a statement, but the discussion suggests that it likely won't be implemented.

    • Note: While PowerShell does have a family of statements starting with keyword using, they serve different purposes - see the conceptual about_Using help topic.

  • A future PowerShell version will support a clean { ... } (or cleanup { ... }) block that is automatically called when an advanced function or script terminates, which allows performing any necessary function-script-level cleanup (disposing of objects) - see RFC #294.

It is up to each type implementing the IDisposable interface whether it calls the .Dispose() methods from the finalizer. Only if so is an object automatically disposed of eventually, by the garbage collector.

For System.IO.StreamWriter and also the lower-level System.IO.FileStream class, this appears not to be the case, so in PowerShell you must call .Close() or .Dispose() explicitly, which is best done from the finally block of a try / catch / finally statement.

You can cut down on the ceremony somewhat by combining the aspects of object construction and variable assignment, but a robust idiom still requires a lot of ceremony:

$x = $y = $null
try {
  ($y = [System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)).
    Write(
      ($x = [System.IO.StreamReader]::new('C:\Temp\a.csv', [System.Text.Encoding]::GetEncoding('iso-8859-1'))).
        ReadToEnd()
    )
} finally {
  if ($x) { $x.Dispose() }
  if ($y) { $y.Dispose() }
}

A helper function, Use-Object (source code below) can alleviate this a bit:

Use-Object 
  ($x = [System.IO.StreamReader]::new('C:\Temp\a.csv',[System.Text.Encoding]::GetEncoding('iso-8859-1'))), 
  ($y = [System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)) `
  { $y.Write($x.ReadToEnd()) }

Use-Object source code:

function Use-Object {
  param( 
    [Parameter(Mandatory)] [array] $ObjectsToDispose,
    [Parameter(Mandatory)] [scriptblock] $ScriptBlock
  )

  try {

    . $ScriptBlock

  } finally {
    foreach ($o in $ObjectsToDispose) {
      if ($o -is [System.IDisposable]) {
        $o.Dispose()
      }
    }
  }
  
}
user_453750
Related