When sending data, why does System.Net.Http.StreamContent read the whole stream into memory, ignoring the given buffer size?

user_62838 0
Viewed 366

The following code reads the whole given Stream's data into memory before sending it as a web request rather than reading and sending the data in smaller blocks. For example, if the stream yields 2 GB of data, the code will read all the 2 GB of data into a memory buffer before it starts sending the request. The 50 KB buffer size passed to StreamContent's constructor seem to be ignored.

using (var httpClient = new HttpClient())
{
  using (var content = new StreamContent(stream, 1024*50)) // 50 KB buffer size ignored?
  {
    using (var request = new HttpRequestMessage(HttpMethod.Post, requestUri))
    {
      request.Content = content;
      await httpClient.SendAsync(request);
    }
  }
}
dotnet-httpclient .net c#
1 Answers

This happens when the given Stream does not support seeking. In that case, the StreamContent cannot use the Stream.Length property to retrieve its size, which is required to construct the HTTP request message.

However, if you specify that the "chunked transfer encoding" should be used, it works. That way, the content can be sent in chunks without having to determine its size up-front.

So, this code will avoid loading the whole data into memory even if it doesn't allow seeking:

using (var httpClient = new HttpClient())
{
  using (var content = new StreamContent(stream, 1024*50))
  {
    using (var request = new HttpRequestMessage(HttpMethod.Post, requestUri))
    {
      request.Content = content;

      // Enable the request message to be sent without knowing the final content size
      request.Headers.TransferEncodingChunked = true;
      
      await httpClient.SendAsync(request);
    }
  }
}
user_628380
Related