Multipart form file upload - Uploaded file have sometimes additional line break at file end

user_4700519 0
Viewed 179

I have a pretty crazy problem and I hope someone can give me some advice on how to fix it.

On the client side, I upload a multipart form to a server using C #. In the form there is a field for an MD5 hash and a field for an audio file.

The problem is that it sometimes happens that an uploaded audio file has a line break at the end of the file that is not present in the original file on the client side. And this of course causes the MD5 hash check to fail.

I am amazed that this only happens sometimes. From 200 requests there is one with the problem described.

Here is the code from my HttpMultipartFormSender class:

public class HttpMultipartFormSender
{
    private HttpRequestMessage _requestMessage;

    public HttpMultipartFormSender(string requestUri)
    {
        UploadProgressHandler = new ProgressMessageHandler();
        var boundary = "----------" + DateTime.Now.Ticks.ToString("x", CultureInfo.InvariantCulture);
        Content = new MultipartFormDataContent(boundary);
        _requestMessage = new HttpRequestMessage(HttpMethod.Post, requestUri);
    }

    public MultipartFormDataContent Content { get; set; }

    public ProgressMessageHandler UploadProgressHandler { get; }

    public async Task<bool> SendFormAsync()
    {
        _requestMessage.Content = Content;
        var client = HttpClientFactory.Create(UploadProgressHandler);
        client.Timeout = TimeSpan.FromMinutes(30);

        var httpResponse = await client.SendAsync(_requestMessage);
        if (!httpResponse.IsSuccessStatusCode)
        {
            throw new InvalidOperationException("Exception thrown while file upload");
        }

        return true;
    }

    #endregion
}

And the usage looks like:

VmPropUserMessage = $"File: {Path.GetFileName(filePath)} is uploading...";
await using var fileStream =
    new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read);

var md5StringContent = new StringContent(GetFileMd5(filePath));
var fileStreamContent = new StreamContent(fileStream);
fileStreamContent.Headers.Add("Content-Type", "application/octet-stream");

var formSender       = new HttpMultipartFormSender($"{VmPropSelectedTranscoder.Url}/upload");
formSender.UploadProgressHandler.HttpSendProgress += FileUploadProgressChanged;
formSender.Content.Add(md5StringContent, "file_hash");
formSender.Content.Add(fileStreamContent, "file", filePath);
await formSender.SendFormAsync();

On the Server side i use this code sniped to store the stream from the request:

private static void SaveFile(TranscodingJobModel transcodingJob, Stream requestFileContent)
{
    var uploadedFilePath = $"{AppConfig.AudioUploadDir}\\{transcodingJob.GetTranscodingJobTmpFileName}";

    using (var fileSaveStream = File.Create(uploadedFilePath))
    {
        requestFileContent.CopyTo(fileSaveStream);
        fileSaveStream.Flush(true);
        fileSaveStream.Close();
        requestFileContent.Close();
    }
}

Sometimes i have this result:

On client side file ends with only one new line:

On source side

On server side file ends with two new lines:

On destination side

Have somebody an idea why this occurs?

Thanks in advance

multipartform-data c#
1 Answers

The multipart content spec has some specific syntax expectations regarding CRLF/new lines:

https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html

Note that the encapsulation boundary must occur at the beginning of a line, i.e., following a CRLF, and that that initial CRLF is considered to be part of the encapsulation boundary rather than part of the preceding part. The boundary must be followed immediately either by another CRLF and the header fields for the next part, or by two CRLFs, in which case there are no header fields for the next part (and it is therefore assumed to be of Content-Type text/plain). NOTE: The CRLF preceding the encapsulation line is considered part of the boundary so that it is possible to have a part that does not end with a CRLF (line break). Body parts that must be considered to end with line breaks, therefore, should have two CRLFs preceding the encapsulation line, the first of which is part of the preceding body part, and the second of which is part of the encapsulation boundary.

The requirement that the encapsulation boundary begins with a CRLF implies that the body of a multipart entity must itself begin with a CRLF before the first encapsulation line -- that is, if the "preamble" area is not used, the entity headers must be followed by TWO CRLFs. This is indeed how such entities should be composed. A tolerant mail reading program, however, may interpret a body of type multipart that begins with an encapsulation line NOT initiated by a CRLF as also being an encapsulation boundary, but a compliant mail sending program must not generate such entities.

The MultipartFormDataContent type inherits the behavior from MultipartContent, where the underlying source helps illustrate how .NET handles applying this spec when combining multiple underlying content elements:

I bring this up because part of the spec involves new lines being added as part of the content segmentation and it may be an opportunity for you to write a unit test on the client with your offending file to generate the MultipartFormDataContent and evaluate it at multiple stages to narrow down where the problem lies and whether that problem is on the client's content generation code or the server's consuming code. If you inspect the file content prior to the multipart content being processed into a stream, does it still match? If you process the multipart content to a stream and read that stream back out locally (without sending it over the wire), does it still match?

user_133742790
Related