I have a spring batch job in which I set the chunk size to 1000 and the reader in that job is a JpaPagingItemReader.
In the reader I set the page size to 20. Does this mean that every chunk of 1000 items read, the reader must take out of the DB 20 items?
If not what is the difference between them?
With your current configuration, if your every read item makes to writer ( i.e. if it doesn't get filtered out in processor ) then you will need 1000/20 = 50 database reads to reach a chunk size i.e. when you actually call the writer for writing.
Spring Batch holds processed items in memory till you reach a chunk size and holding items costs memory.
Your current configuration is holding data in memory and making unnecessary database calls while we wish to reduce both of those things.
So your configuration needs to be reverse of what you are doing i.e. increase reader page size to a minimum equal to chunk size / commit interval or more so data once read gets processed in small small chunks then you go again and read database .
So as you have noticed till current write up, conceptually these are unrelated concepts - reader page size is to minimize database calls ( and this concept is not a spring batch concept but reader specific - if its not a paging reader, this concept doesn't come into picture ) while chunk size is about committing processed data in small small chunks to reduce memory foot print.
Yes the commit interval determines how many record would be processed in a Chunk.
The database page size determines how many record would be fetched from the database in one go. It's more of an optimization setting between how big the buffer would you like to have v/s the number of trips the driver would how to make to fetch the data from the database.
