FileStream corrupt zip - SignalR Stream

user_568405 0
Viewed 147

I'm attempting to take a zip file created on the client, and upload that file in chunks via a Hub stream. When the stream ends, and I try to open the file, windows states it can't open, {file name} is invalid.

The byte array, prior to chunking, writes out fine on the client when doing a File.WriteAllBytes(path, bytes[]). File sizes are identical when looking at the streamed file, and the File.WriteAllBytes.

I'm out of ideas...

Client side

private async IAsyncEnumerable<byte[]> StreamBytes(byte[] bytes)
    {
        //this works and I'm able to open the file it creates.
        File.WriteAllBytes(@"C:\test.zip", bytes);
        long fileSize = bytes.Length;
        long fileWrite = 0;

        while (fileSize != 0)
        {
            byte[] buffer = fileSize > ByteHelper.BUFFER_SIZE ? new byte[ByteHelper.BUFFER_SIZE] : new byte[fileSize];
            Array.Copy(bytes, buffer, buffer.Length);

            fileSize -= buffer.Length;
            fileWrite += buffer.Length;

            var bufResult = await Task.FromResult(buffer);
            yield return bufResult;
        }
    }

Server Side

public async Task UploadCommandResultStream(IAsyncEnumerable<byte[]> byteChunk, int commandQueueKey, string folder)
    {
        var path = Path.Combine(_settings.UploadPath, folder);
        if (Directory.Exists(path) == false)
        {
            Directory.CreateDirectory(path);
        }

        var file = Path.Combine(path, $"{commandQueueKey}.zip");
        long bytesread = 0;
        try
        {
            //this creates a file but can't open
            using (FileStream fs = new FileStream(file, FileMode.Create, FileAccess.Write, FileShare.Read, ByteHelper.BUFFER_SIZE))
            {
                await foreach (var bytes in byteChunk)
                {
                    fs.Write(bytes, 0, bytes.Length);
                    bytesread += bytes.Length;
                }
            }
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "asyncwrite");
        }
    }

enter image description here enter image description here

signalr .net-core c#
3 Answers

Probably, you'll need to change your approach; the new proposed feature Async Streams in C# 8 makes the async pattern more flexible, but I don't think this approach ends or flushs the FileStream object properly. Please, try to move last method with this syntax:

//this creates a file but can't open
using (FileStream fs = new FileStream(file, FileMode.Create, FileAccess.Write, FileShare.Read, ByteHelper.BUFFER_SIZE, true))
{
   await fs.WriteAsync(bytes, 0, bytes.Length);
}
user_105497910

I believe your file just isn't flushing the last buffer out fully. Try changing to this

//this creates a file but can't open
using (FileStream fs = new FileStream(file, FileMode.Create, FileAccess.Write, FileShare.Read, ByteHelper.BUFFER_SIZE))
{
    await foreach (var bytes in byteChunk)
    {
        fs.Write(bytes, 0, bytes.Length);
        bytesread += bytes.Length;
    }
    fs.Flush(); //or await fs.FlushAsync()
}
user_24525050

OK I figured out the answer. It was the IAsyncEnumerable on the client side. As a test I wrote the file to disk on client side, which I knew worked, then opened a FileStream and read in chunks yielding the results.

Still investigating why original client code was wrong, but this 100% works now.

The code needs cleaned up a bit as the trimming buffer could be handled a bit better but just wrote it real quick.

Client Side

private async IAsyncEnumerable<byte[]> StreamBytes(byte[] bytes)
    {
        string fileName = @"test.zip";
        File.WriteAllBytes(fileName, bytes);

        using(var fs = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read, ByteHelper.BUFFER_SIZE, true))
        {
            long fileSize = fs.Length;
            int totalBytesRead = 0;
            byte[] buffer = new byte[ByteHelper.BUFFER_SIZE];

            while (fileSize > 0)
            {
                fs.Seek(totalBytesRead, SeekOrigin.Begin);
                var bytesRead = await fs.ReadAsync(buffer, 0, ByteHelper.BUFFER_SIZE);
                if (bytesRead == 0)
                    break;

                totalBytesRead += bytesRead;
                fileSize -= bytesRead;

                if (bytesRead != ByteHelper.BUFFER_SIZE)
                {
                    byte[] trimmedBuffer = new byte[bytesRead];
                    Array.Copy(buffer, trimmedBuffer, bytesRead);

                    buffer = new byte[bytesRead];
                    Array.Copy(trimmedBuffer, buffer, bytesRead);
                }

                yield return buffer;
            }
        }
        //long fileSize = bytes.Length;
        //long fileWrite = 0;

        //while (fileSize != 0)
        //{
        //    byte[] buffer = fileSize > ByteHelper.BUFFER_SIZE ? new byte[ByteHelper.BUFFER_SIZE] : new byte[fileSize];
        //    Array.Copy(bytes, buffer, buffer.Length);

        //    fileSize -= buffer.Length;
        //    fileWrite += buffer.Length;

        //    var bufResult = await Task.FromResult(buffer);
        //    yield return bufResult;
        //}
    }
user_5684050
Related