Issue with Azure chunked upload to fileshare via Azure.Storage.Files.Shares library

user_1736797 0
Viewed 1753

I'm trying to upload files to an Azure fileshare using the library Azure.Storage.Files.Shares.

If I don't chunk the file (by making a single UploadRange call) it works fine, but for files over 4Mb I haven't been able to get the chunking working. The file is the same size when downloaded, but won't open in a viewer.

I can't set smaller HttpRanges on a large file as I get a 'request body is too large' error, so I'm splitting the filestream into multiple mini streams and uploading the entire HttpRange of each of these

        ShareClient share = new ShareClient(Common.Settings.AppSettings.AzureStorageConnectionString, ShareName());
        ShareDirectoryClient directory = share.GetDirectoryClient(directoryName);

        ShareFileClient file = directory.GetFileClient(fileKey);
        using(FileStream stream = fileInfo.OpenRead())
        {
            file.Create(stream.Length);

            //file.UploadRange(new HttpRange(0, stream.Length), stream);

            int blockSize = 128 * 1024;

            BinaryReader reader = new BinaryReader(stream);
            while(true)
            {
                byte[] buffer = reader.ReadBytes(blockSize);
                if (buffer.Length == 0)
                    break;

                MemoryStream uploadChunk = new MemoryStream();
                uploadChunk.Write(buffer, 0, buffer.Length);
                uploadChunk.Position = 0;

                file.UploadRange(new HttpRange(0, uploadChunk.Length), uploadChunk);
            }

            reader.Close();
        }

The code above uploads without error, but when downloading the image from Azure it is corrupt.

Does anyone have any ideas? Thanks for any help you can provide.

cheers

Steve

fileshare azure
1 Answers

I was able to reproduce the issue. Basically the problem is with the following line of code:

new HttpRange(0, uploadChunk.Length)

Essentially you're always setting the content at the same range and that's why the file is getting corrupted.

Please try the code below. It should work. What I did here is defined the HTTP range offset and moving it constantly with number of bytes already written to the file.

        using (FileStream stream = fileInfo.OpenRead())
        {
            file.Create(stream.Length);

            //file.UploadRange(new HttpRange(0, stream.Length), stream);

            int blockSize = 1 * 1024;
            long offset = 0;//Define http range offset
            BinaryReader reader = new BinaryReader(stream);
            while (true)
            {
                byte[] buffer = reader.ReadBytes(blockSize);
                if (buffer.Length == 0)
                    break;

                MemoryStream uploadChunk = new MemoryStream();
                uploadChunk.Write(buffer, 0, buffer.Length);
                uploadChunk.Position = 0;

                HttpRange httpRange = new HttpRange(offset, buffer.Length);
                var resp = file.UploadRange(httpRange, uploadChunk);
                offset += buffer.Length;//Shift the offset by number of bytes already written
            }

            reader.Close();
        }
user_1880960
Related