I'm using JpaPagingItemReader with Spring batch job to read the data from Database. I have a requirement where I can only write 10k records to external server in each try. I set chunk size and page size to 10000. Will this help to read, process and write only 10k records in each try until the JpaPagingItemReader is exhausted with response null and all the data from DB is read? I'm confused between Pagesize and setMaxItemCount values. Which method will ensure to read the 10k records from DB?
Batch config Step:
public Step step1(StepBuilderFactory stepBuilderFactory, @Qualifier("itemReader") ItemReader<BatchRecords> ir,
@Qualifier("itemProcessor") ItemProcessor<BatchRecords, Map<String, Object>> ip,
@Qualifier("itemWriter") ItemWriter<Map<String, Object>> iw) throws Exception {
logger.info("Step1");
return stepBuilderFactory.get("step1").<BatchRecords, Map<String, Object>>chunk(10000).reader(ir).processor(ip)
.writer(iw).build();
}
ItemReader:
@Bean(destroyMethod="")
@StepScope
public JpaPagingItemReader<BatchRecords> itemReader(){
Map<String, Object> dates = batchUtil.dateFormat();
String startDate = (String) dates.get("StartDate");
String endDate = (String) dates.get("EndDate");
logger.info("Batch settlement Start date : {}, End Date: {}",startDate,endDate);
JpaPagingItemReader<BatchRecords> reader = new JpaPagingItemReader<BatchRecords>();
reader.setEntityManagerFactory(localContainerdbEntityManagerFactory.getNativeEntityManagerFactory());
reader.setQueryString("select b from BatchRecords b where b.status = 'O' and b.ordTimestamp between '" + startDate + "' and '" + endDate+ "' ");
reader.setPageSize(10000);
// reader.setMaxItemCount(50*reader.getPageSize());
try {
reader.afterPropertiesSet();
} catch (Exception e) {
logger.error("Exception in paging Item reader: "+e.getMessage());
}
return reader;
}```