POST gzip to influxdb results in "unexpected EOF"

user_2056222 0
Viewed 157

Trying to write a gzip post request to influxdb, but getting "unexpected EOF"

The following code is in F#, but it is straightforward. You can translate it easily in your mind to C#

let gzip(s: string) =
    let bytes = Encoding.UTF8.GetBytes s
    use ms = new MemoryStream()
    use gz = new GZipStream(ms, CompressionLevel.Fastest)
    (gz.Write(bytes, 0, bytes.Length))
    ms.ToArray()
let toGzipContent(s: string) =
    let content = new ByteArrayContent(gzip s)
    content.Headers.ContentEncoding.Add "gzip"
    content.Headers.ContentType <- MediaTypeHeaderValue("text/plain", CharSet = "utf-8")
    content
let postTest() =
    let client = new HttpClient()

    let hash = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{cfg.user}:{cfg.password}"))
    client.DefaultRequestHeaders.Authorization <- AuthenticationHeaderValue("Basic", hash )
    client.DefaultRequestHeaders.ConnectionClose <- false

    let content =
        "d1fcd8ad-1dea-4a91-b1ed-eafffd497ade,tag=tag0 field=0.5177723544266878 1626025134320"
        |> toGzipContent
    let resp = client.PostAsync(cfg.url, content)
    resp

The influxdb response:

  StatusCode: 400, ReasonPhrase: 'Bad Request', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
  Request-Id: a1edc900-e26e-11eb-b78b-244bfe8c4492
  X-Influxdb-Build: OSS
  X-Influxdb-Error: unexpected EOF
  X-Influxdb-Version: 1.8.6
  X-Request-ID: a1edc900-e26e-11eb-b78b-244bfe8c4492
  Date: Sun, 11 Jul 2021 17:37:15 GMT
  Content-Type: application/json
  Content-Length: 27
}

Sending plain text works, but gzipped not. What could be wrong here? Already looked into influxdb .NET client implementations and found no issues

influxdb post gzip f# c#
2 Answers

In the end, I have just misplaced the braces:

let gzip(s: string) =
  let bytes = Encoding.UTF8.GetBytes s
  use ms = new MemoryStream()
  ( use gz = new GZipStream(ms, CompressionLevel.Fastest)
    gz.Write(bytes, 0, bytes.Length)
  )
  ms.ToArray()

More elaborated, the devil is in the kind of how and when the nested usings are disposed. In F# since ever and in C# just since the latest version where you are allowed to write this:

public byte[] gzip(s: string)
{
    var bytes = Encoding.UTF8.GetBytes(s);
    using var ms = new MemoryStream();
    using (var gz = new GZipStream(ms, CompressionLevel.Fastest))
    {
        gz.Write(bytes, 0, bytes.Length);
        // the most nested block, writing to the array, should explicitly end here
        // to let gz be disposed and flushed
    }
    return ms.ToArray();
}

Like @CaringDev answered, the gz stream was not flushed yet, when the output array was created.

user_20562220

The gzipped content is empty, the stream needs to be flushed. Inserting gz.Flush() between writing and converting the underlying ms.ToArray() should fix this.

user_28947700
Related