GKE + HTTPS downloading large file fails with "backend_connection_closed_after_partial_response_sent"

Viewed 126

The Problem is this:
Making available large files (up to ~20 GB) is one of the primary use cases of the app (created on the backend). Downloading fails for large files with the browser simply stating: "Failed - Network error".
This happens on my work PC, various co-workers that are in home-office, my private PC - so the client is not causing the issue, browser choice also does not matter.

Time elapsed and amount of data transferred does not matter - example (8GB file):
70s / 3037963535
126s / 5646718607
109s / 4642356851
The time to fail can even be 5+ minutes if the connection is slow

  • In the TEST environment that is identical to PROD except for lacking a domain/TLS it works no matter what size the file is and also if the time to completion is about 10 minutes

The app is running on GKE:

  • Ingress with TLS

  • Python / Flask / Waitress backend

  • nginx / Angular frontend

  • The log from the LoadBalancer gives me this message:

     jsonPayload: { 
        @type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
        statusDetails: "backend_connection_closed_after_partial_response_sent"
     }
    

Googling the error message gives me no clues on how to fix this. There are some mentions of WebSocket being the cause but this is a generic download, I've simplified it down to a window.open(BACKEND-URL) while hunting the cause. Also the timeout values usually applied to WebSocket do not fit.

Looking at the Ingress spec there does not seem to be anything to configure that could cause/resolve this.

0 Answers
Related