How do I use GZipStream with System.IO.MemoryStream?

Viewed 84015

I am having an issue with this test function where I take an in memory string, compress it, and decompress it. The compression works great, but I can't seem to get the decompression to work.

//Compress
System.IO.MemoryStream outStream = new System.IO.MemoryStream();                
GZipStream tinyStream = new GZipStream(outStream, CompressionMode.Compress);
mStream.Position = 0;
mStream.CopyTo(tinyStream);

//Decompress    
outStream.Position = 0;
GZipStream bigStream = new GZipStream(outStream, CompressionMode.Decompress);
System.IO.MemoryStream bigStreamOut = new System.IO.MemoryStream();
bigStream.CopyTo(bigStreamOut);

//Results:
//bigStreamOut.Length == 0
//outStream.Position == the end of the stream.

I believe that bigStream out should at least have data in it, especially if my source stream (outStream) is being read. is this a MSFT bug or mine?

10 Answers

If you still need it, you can use GZipStream constructor wit boolean argument (there are two such constructors) and pass true value there:

tinyStream = new GZipStream(outStream, CompressionMode.Compress, true);

In that case, when you close your tynyStream, your out stream will be still opened. Don't forget to copy data:

mStream.CopyTo(tinyStream);
tinyStream.Close();

Now you've got memory stream outStream with zipped data

Bugs and kisses for U

Good luck

I had an issue where *.CopyTo(stream)* would end up with a byte[0] result. The solution was to add .Position=0 before calling .CopyTo(stream) Answered here

I also use a BinaryFormatter that would throw an 'End of stream encountered before parsing was completed' exception if position was not set to 0 before deserialization. Answered here

This is the code that worked for me.

 public static byte[] SerializeAndCompressStateInformation(this IPluginWithStateInfo plugin, Dictionary<string, object> stateInfo)
    {
        byte[] retArr = new byte[] { byte.MinValue };
        try
        {
            using (MemoryStream msCompressed = new MemoryStream())//what gzip writes to
            {
                using (GZipStream gZipStream = new GZipStream(msCompressed, CompressionMode.Compress))//setting up gzip
                using (MemoryStream msToCompress = new MemoryStream())//what the settings will serialize to
                {
                    BinaryFormatter formatter = new BinaryFormatter();
                    //serialize the info into bytes
                    formatter.Serialize(msToCompress, stateInfo);
                    //reset to 0 to read from beginning byte[0] fix.
                    msToCompress.Position = 0;
                    //this then does the compression
                    msToCompress.CopyTo(gZipStream);
                }
                //the compressed data as an array of bytes
                retArr = msCompressed.ToArray();
            }
        }
        catch (Exception ex)
        {
            Logger.Error(ex.Message, ex);
            throw ex;
        }
        return retArr;
    }


    public static Dictionary<string, object> DeserializeAndDecompressStateInformation(this IPluginWithStateInfo plugin, byte[] stateInfo)
    {
        Dictionary<string, object> settings = new Dictionary<string, object>();
        try
        {

            using (MemoryStream msDecompressed = new MemoryStream()) //the stream that will hold the decompressed data
            {
                using (MemoryStream msCompressed = new MemoryStream(stateInfo))//the compressed data
                using (GZipStream gzDecomp = new GZipStream(msCompressed, CompressionMode.Decompress))//the gzip that will decompress
                {
                    msCompressed.Position = 0;//fix for byte[0]
                    gzDecomp.CopyTo(msDecompressed);//decompress the data
                }
                BinaryFormatter formatter = new BinaryFormatter();
                //prevents 'End of stream encountered' error
                msDecompressed.Position = 0;
                //change the decompressed data to the object
                settings = formatter.Deserialize(msDecompressed) as Dictionary<string, object>;
            }
        }
        catch (Exception ex)
        {
            Logger.Error(ex.Message, ex);
            throw ex;
        }
        return settings;
    }

I thought I would share this answer for anyone interested in reproducing this on PowerShell, the code is mostly inspired from Timwi's helpful answer, however unfortunately as of now there is no implementation for the using statement like on C# for PowerShell, hence the need to manually dispose the streams before output.

Functions below requires PowerShell 5.0+.

  • Compression from string to Base64 GZip compressed string:
using namespace System
using namespace System.Text
using namespace System.IO
using namespace System.IO.Compression

function Compress-GzipString {
[cmdletbinding()]
param(
    [Parameter(Mandatory, ValueFromPipeline)]
    [string]$String,
    [string]$Encoding = 'UTF8'
)

    try {
        $outStream = [MemoryStream]::new()
        $gzip = [GZipStream]::new(
            $outStream,
            [CompressionMode]::Compress,
            [CompressionLevel]::Optimal
        )
        $inStream = [MemoryStream]::new(
            [Encoding]::$Encoding.GetBytes($string)
        )
        $inStream.CopyTo($gzip)
    }
    catch {
        $PSCmdlet.WriteError($_)
    }
    finally {
        ($gzip, $outStream, $inStream).ForEach('Dispose')
    }
    try {
        [Convert]::ToBase64String($outStream.ToArray())
    }
    catch {
        $PSCmdlet.WriteError($_)
    }
}
  • Expansion from Base64 GZip compressed string to string:
function Expand-GzipString {
[cmdletbinding()]
param(
    [Parameter(Mandatory, ValueFromPipeline)]
    [string]$String,
    [string]$Encoding = 'UTF8'
)

    try {
        $bytes = [Convert]::FromBase64String($String)
        $outStream = [MemoryStream]::new()
        $inStream = [MemoryStream]::new($bytes)
        $gzip = [GZipStream]::new(
            $inStream,
            [CompressionMode]::Decompress
        )
        $gzip.CopyTo($outStream)
        [Encoding]::$Encoding.GetString($outStream.ToArray())
    }
    catch {
        $PSCmdlet.WriteError($_)
    }
    finally {
        ($gzip, $outStream, $inStream).ForEach('Dispose')
    }
}

And for the little Length comparison, querying the Loripsum API:

$loremIp = Invoke-RestMethod loripsum.net/api/10/long
$compressedLoremIp = Compress-GzipString $loremIp

$loremIp, $compressedLoremIp | Select-Object Length

Length
------
  8353
  4940

(Expand-GzipString $compressedLoremIp) -eq $loremIp # => Should be True
Related