XMLEventReader from ZipInputStream closes stream

user_3448078 0
Viewed 234

I want to stream a ZIP file containing several very large (~1GByte) XML files. I could read the data from each zip file into a buffer and create a XMLStream from that - but to save on memory I would prefer to process the data on the fly.

@Test
public void zipStreamTest() throws IOException, XMLStreamException {
    FileInputStream fis = new FileInputStream("archive.zip");
    ZipInputStream zis = new ZipInputStream(fis);
    ZipEntry ei;

    while ((ei = zis.getNextEntry()) != null){      
        XMLEventReader xr = XMLInputFactory.newInstance().createXMLEventReader(zis);
        while (reader.hasNext()) {
            XMLEvent xe = xr.nextEvent();
            // do some xml event processing..
        }
        zis.closeEntry();
    }
    zis.close();
}

The problem: I'm getting a java.io.IOException: Stream closed when executing zis.closeEntry();. When I remove that line, the same error is thrown at zis.getNextEntry() which closes previous entries if they're still open automatically.

It seems that my XML stream reader is breaking the stream at the end of the XML file so that the rest of the zip can't be processed.

Do I have an implementation error or is my conception of how streams work incorrect?

Note: To make this a minimal reproduceable example all you need is a zip file "archive.zip" which contains any valid XML file (no subdirectories inside the zip!). You can then run the snippet using JUnit.

zip xml stream java
2 Answers

You could try to open separate InputStream for each entry using java.util.zip.ZipFile:

@Test
public void zipStreamTest() throws Exception {
        ZipFile zipFile = new ZipFile("archive.zip");
        Iterator<? extends ZipEntry> iterator = zipFile.entries().asIterator();
        while (iterator.hasNext()) {
            ZipEntry ze = iterator.next();
            try (InputStream zis = zipFile.getInputStream(ze)) {
                XMLEventReader reader = XMLInputFactory.newInstance().createXMLEventReader(zis);
                while (reader.hasNext()) {
                    XMLEvent xe = reader.nextEvent();
                    // do some xml event processing
                }
                reader.close();
            }
        }
    }
user_141686230

I would recommend using ZipFile instead of ZipInputStream, as suggested in answer by Alexandra Dudkina.

However, if you're processing a data stream while e.g. downloading, and therefore want to keep using ZipInputStream, you should wrap it in a CloseShieldInputStream from Apache Commons IO1 inside the getNextEntry() loop:

while ((ei = zis.getNextEntry()) != null) {
    XMLEventReader xr = XMLInputFactory.newInstance().createXMLEventReader(new CloseShieldInputStream(zis));
    // Process XML here
    zis.closeEntry();
}

1) Or other similar helper class from third-party library of your choice.

user_52211490
Related