Spring Integration UnZipTransformer output. How to access the files in the subdirectory

user_5244350 0
Viewed 97

I'm new to Spring Integration and I have a project where I want to process the contents of a zip file. The zip contains n number of tiff files and one xml file. The xml file contains information about how many tiff files should be in the zip so I need to compare the number of tiff files with the info in the xml file.

My process is: poll a directory for a zip file, move the zip file to a "working" directory, unzip the file, find the xml file and read the contents, count the tiff files and confirm that the names and number of files matches the xml data.

I have all of the steps working except for the stage where I try to read the zip and tiff files. My problem is that the UnZipTransformer creates an intermediate directory with a name in uuid format below the working directory and I'm unable to work out how to get the files from this directory.

directory structure after the unzip has happened

working/
   0eca3f50-eedb-9ebd-5a3a-4ceb3ad8abbd/
        01.tif
        02.tif
        my.xml

This Flow works. It extracts the contents of the zip file.

@Configuration
public class FileUnzipIntegrationFlow {

    public static final String UNZIP_OUTPUT = "unzipOutputChannel";
    @Value("${unzipped.dir}")
    String unzippedDir;
    @Value("${working.dir}")
    String workingDir;

    @Bean
    public MessageSource<File> unzippedDirectory() {
        FileReadingMessageSource messageSource = new FileReadingMessageSource();
        messageSource.setDirectory(new File(unzippedDir));
        return messageSource;
    }


    @Bean
    public IntegrationFlow fileUnzipper() {
        return IntegrationFlows.from(unzippedDirectory(), c -> c.poller(Pollers.fixedDelay(1000)))
                .filter(source -> ((File)source).getName().endsWith(".zip"))
                .transform(unZipTransformer())
                .log()
                .get();
    }

    @Bean(name = UNZIP_OUTPUT)
    public SubscribableChannel unzipOutputChannel()
    {
        return MessageChannels.publishSubscribe(UNZIP_OUTPUT)
                .get();
    }


    @Bean
    @Transformer(inputChannel = "input", outputChannel = UNZIP_OUTPUT)
    public UnZipTransformer unZipTransformer() {
        UnZipTransformer unZipTransformer = new UnZipTransformer();
        unZipTransformer.setExpectSingleResult(false);
        unZipTransformer.setZipResultType(ZipResultType.FILE);
        unZipTransformer.setWorkDirectory(new File(workingDir));
        unZipTransformer.setDeleteFiles(false);
        return unZipTransformer;
    }

}

I can't work out how to get to the sub-directory in this Flow

import static com.santanderuk.spring.integration.FileUnzipIntegrationFlow.UNZIP_OUTPUT;

@Configuration
public class XmlVerificationFlow {
    @Value("${working.dir}")
    String workingDir;

    @Bean
    public IntegrationFlow xmlVerfier() {
        return IntegrationFlows.from(xmlWorkingDirectory(),
                        c -> c.poller(Pollers.fixedRate(1000).maxMessagesPerPoll(1))).
                               filter(source -> ((File)source).getName().toLowerCase().endsWith(".xml")).

                handle(xmlFileHandler()).
                get();
    }


    @Bean
    public MessageSource<File> workingDirectory() {
        FileReadingMessageSource messageSource = new FileReadingMessageSource();
        messageSource.setDirectory(new File(workingDir));
        return messageSource;
    }

The snippet above only works when I manually move the xml file from the sub-directory into the working directory. I can also see the payload value in the logging, which has the directory name I need but I have not been able to find out how to access this information

2022-03-28 19:39:14.395  INFO 7588 --- [   scheduling-1] o.s.integration.handler.LoggingHandler   : GenericMessage [payload={10000001.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000001.tif, 10000002.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000002.tif, 10000003.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000003.tif, 10000004.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000004.tif, 10000005.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000005.tif, 10000006.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000006.tif, 10000007.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000007.tif, 10000008.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000008.tif, 10000009.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000009.tif, 10000010.tif=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\10000010.tif, 20220304S092800.XML=processing\working\1cd8f803-2e45-dfe2-1c39-99b0d74f83f0\20220304S092800.XML}, headers={file_originalFile=processing\unzipped\202203040001.zip, id=2835eb9e-ff3b-71bf-7432-4967a1f808f6, file_name=202203040001.zip, file_relativePath=202203040001.zip, timestamp=1648492754392}]
spring-integration java
2 Answers

It looks like you just ignore the power of the UnZipTransformer. It does return for us something like this:

final SortedMap<String, Object> uncompressedData = new TreeMap<>();

which has a content like this in your case:

uncompressedData.put(zipEntryName, destinationFile);

So, you don't need that extra flow to poll a working dir when you can work easily with the result of unzipping.

On the other hand the FileReadingMessageSource can be configured with the RecursiveDirectoryScanner to let it to iterate sub-dirs recursively. By default it scans only the top dir and ignore all the sub-dirs.

user_27565470

Thanks very much Artem. As suggested, I used the SortedMap that was returned by the UnZipTransformer in a ZipOutputHandler to count the tiff files and record the xml file name.

Updated Flow

    @Bean
    public IntegrationFlow fileUnzipper() {
        return IntegrationFlows.from(unzippedDirectory(), c -> c.poller(Pollers.fixedDelay(1000)))
                .filter(source -> ((File) source).getName().endsWith(".zip"))
                .transform(unZipTransformer())
                .handle(zipOutputHandler(), "process")
                .log()
                .get();
    }

process method in the Handler

    public File process(SortedMap<String, Object> uncompressedData) {
        uncompressedData.forEach((s, o) -> {
            if (s.endsWith("tif")) {
                tiffCount++;
                tiffNames.add(s);
            }
            if (s.endsWith("XML")) {
                xmlName = s;
                xmlFile = new File(o.toString());
                extractedDirectory = xmlFile.getParentFile();
            }
        });
user_52443500
Related