Google App Engine - how to gzip requests correctly

Question

In my Google App Engine app (Standard Environment, written in Java + Scala) I want some of my requests to the server to be gzipped. After a bit of experimenting I got it mostly working, but there are few points which I am uncertain about. I did not find much documentation about correct client-side gzip usage, most documentation and examples seem to be concerned about server encoding its responses, therefore I am unsure if I am doing everything as I should.

I send the request this way (using akka.http in the client application):

        val uploadReq = Http().singleRequest(
          HttpRequest(
            uri = "https://xxx.appspot.com/upload-a-file",
            method = HttpMethods.POST,
            headers = List(headers.`Content-Encoding`(HttpEncodings.gzip)) 
            entity = HttpEntity(ContentTypes.`text/plain(UTF-8)`,  Gzip.encode(ByteString(bytes)))
          )
        )

On a production GAE server, I get the gzipped request body already decoded, with the encoding header still present. On a development server this is different, the header is also present, but the request body is still gzipped.

The code for decoding the request input stream is not a problem, but I did not find a clean way how to check in my server code if I should decode the request body or not. My current workaround is that if the client knows it is communicating with the development server, it does not use gzip encoding at all, and I never attempt to decode the request body, as I rely upon the Google App Engine to do this for me.

should I encode the request body differently on the client?
is there some other way to recognize on the server if the incoming request body needs decoding or not?
may I assume Google App Engine production servers will decode the body for me?

user_16673 · Answer

For the record: the solution I have ended up with is that I check the request body and if it looks like gzipped, I unzip it, ignoring the header completely. This works both on prod (where App Engine does the unzipping and the code does no harm) and dev (where the code unzips). Scala code follows:

def decompressStream(input: InputStream): InputStream = {
  val pushbackInputStream = new PushbackInputStream(input, 2)
  val signature = new Array[Byte](2)
  pushbackInputStream.read(signature)
  pushbackInputStream.unread(signature)
  if (signature(0) == 0x1f.toByte && signature(1) == 0x8b.toByte) {
    new GZIPInputStream(pushbackInputStream)
  } else pushbackInputStream
}

The theoretical drawback is someone could send a request which contains 0x1f/0x8b header just by chance. This cannot happen in my case, therefore I am fine with it.

Google App Engine - how to gzip requests correctly

1 Answers