Servlet buffering response despite calls to flush()

user_884373 0
Viewed 11362

We have a system where a client makes an HTTP GET request, the system does some processing on the backend, zips the results, and sends it to the client. Since the processing can take some time, we send this as a ZipOutputStream wrapping the response.getOutputStream().

However, when we have an exceptionally small amount of data in the first ZipEntry, and the second entry takes a long time, the browser the client is using times out. We've tried flushing the stream buffer, but no response seems to be sent to the browser until at least 1000 bytes have been written to the stream. Oddly, once the first 1000 bytes have been sent, subsequent flushes seem to work fine.

I tried stripping down the code to bare-bones to give an example:

protected void doGet(HttpServletRequest request,
        HttpServletResponse response) throws ServletException, IOException {
    try {
        ZipOutputStream _zos = new ZipOutputStream( response.getOutputStream());
        ZipEntry _ze = null;
        long startTime = System.currentTimeMillis();
        long _lByteCount = 0;

        response.setContentType("application/zip");

        while (_lByteCount < 2000) {
            _ze = new ZipEntry("foo");
            _zos.putNextEntry( _ze );

            //writes 100 bytes and then waits 10 seconds
            _lByteCount += StreamWriter.write( 
                    new ByteArrayInputStream(DataGenerator.getOutput().toByteArray()),
                    _zos );
            System.out.println("Zip: " + _lByteCount + " Time: " + ((System.currentTimeMillis() - startTime) / 1000));

            //trying to flush
            _zos.finish();
            _zos.flush();
            response.flushBuffer();
            response.getOutputStream().flush();
        }
    } catch (Throwable e) {
        e.printStackTrace();
    }
}

I set my browser timeout to be about 20 seconds for easy reproduction. Despite writing the 100 bytes a couple of times, nothing is sent to the browser and the browser times out. If I expand the browser timeout, nothing gets sent until 1000 bytes have been written and then the browser pops up the "Would you like to save..." dialog. Again, after the initial 1000 bytes, each addition 100 bytes sends fine, rather than buffering to 1000 byte chunks.

If I set the max byte count in the while condition to 200 or so, it works fine, sending only 200 bytes.

What can I do to force the servlet to send back really small initial amounts of data?

servlets tomcat java
6 Answers

I know this is a really, really old question, but for the record, I wanted to post an answer that should be a fix all for the issue that you are experiencing.

The key is that you want to flush the response stream, not the zip stream. Because the ZIP stream cannot flush what is not yet ready to write. Your client, as you mentioned, is timing out because it is not receiving a response in a predetermined amount of time, but once it receives data, it is patient and will wait a very long time to download the file, thus the fix is easy, provided you flush the correct stream. I recommend the following:

protected void doGet(HttpServletRequest request,
    HttpServletResponse response) throws ServletException, IOException {
try {
    ZipOutputStream _zos = new ZipOutputStream( response.getOutputStream());
    ZipEntry _ze = null;
    long startTime = System.currentTimeMillis();
    long _lByteCount = 0;

    response.setContentType("application/zip");
    // force an immediate response of the expected content
    // so the client can begin the download process
    response.flushBuffer();

    while (_lByteCount < 2000) {
        _ze = new ZipEntry("foo");
        _zos.putNextEntry( _ze );

        //writes 100 bytes and then waits 10 seconds
        _lByteCount += StreamWriter.write( 
                new ByteArrayInputStream(DataGenerator.getOutput().toByteArray()),
                _zos );
        System.out.println("Zip: " + _lByteCount + " Time: " + ((System.currentTimeMillis() - startTime) / 1000));

        //trying to flush
        _zos.finish();
        _zos.flush();
    }
} catch (Throwable e) {
    e.printStackTrace();
}

Now, what should happen here, is the header and response codes will be committed along with anything in the response buffer's OutputStream. This does not close the stream, so any additional writes to the stream are appended. The downside to doing it this way, is that you cannot know the content-length to assign to the header. The positive is that you are starting the download immediately, and not allowing the browser to timeout.

user_15753100
Related