Spring batch JpaPagingItemReader page size and maxitemcount

Viewed 1558

I'm using JpaPagingItemReader with Spring batch job to read the data from Database. I have a requirement where I can only write 10k records to external server in each try. I set chunk size and page size to 10000. Will this help to read, process and write only 10k records in each try until the JpaPagingItemReader is exhausted with response null and all the data from DB is read? I'm confused between Pagesize and setMaxItemCount values. Which method will ensure to read the 10k records from DB?

Batch config Step:

    public Step step1(StepBuilderFactory stepBuilderFactory, @Qualifier("itemReader") ItemReader<BatchRecords> ir,
            @Qualifier("itemProcessor") ItemProcessor<BatchRecords, Map<String, Object>> ip,
            @Qualifier("itemWriter") ItemWriter<Map<String, Object>> iw) throws Exception {
        logger.info("Step1");
        return stepBuilderFactory.get("step1").<BatchRecords, Map<String, Object>>chunk(10000).reader(ir).processor(ip)
                .writer(iw).build();

    }



ItemReader:

    @Bean(destroyMethod="")
    @StepScope
    public JpaPagingItemReader<BatchRecords> itemReader(){
        Map<String, Object> dates = batchUtil.dateFormat();
        String startDate = (String) dates.get("StartDate");
        String endDate = (String) dates.get("EndDate");
        logger.info("Batch settlement Start date : {}, End Date: {}",startDate,endDate);
        JpaPagingItemReader<BatchRecords> reader = new JpaPagingItemReader<BatchRecords>();
        reader.setEntityManagerFactory(localContainerdbEntityManagerFactory.getNativeEntityManagerFactory());
        reader.setQueryString("select b from BatchRecords b where b.status = 'O' and b.ordTimestamp between '" + startDate + "' and '" + endDate+ "' ");

        reader.setPageSize(10000);
//      reader.setMaxItemCount(50*reader.getPageSize());
        try {
            reader.afterPropertiesSet();
        } catch (Exception e) {
            logger.error("Exception in paging Item reader: "+e.getMessage());
        }
        return reader;

    }```
1 Answers

You need to use maxItemCount if you want to limit the total number of items to read. I think it's even better to set the limit in you SQL query. PageSize will tell the reader how many items to read in each page (which is not a limit on the total number of items but a limit on the number of items to read when a new page is requested).

Related