There is a requirement where I need to read and process data fetched from a rest api let say restApi1 and write to different rest api let say restApi2 . For this I am using chunk oriented approach . But the issue is currently the restApi1 is not paginated . That endpoint returns a large number of data approximately 10000 .
So if my step failed then while restarting I have to read all the data again and process .
I can not start from where it failed .
Is this thought correct in relation to spring batch processing ?
Kindly suggest some possible approach .
ItemReader
public class MyItemReader extends ItemStreamSupport implements ItemReader<Data> {
private int curIndex = 0;
@Override
public void open(ExecutionContext executionContext) {
this.curIndex = 0;
}
@Override
public void update(ExecutionContext executionContext) {
}
}
ItemProcessor
public class MyItemProcessor extends ItemStreamSupport implements ItemProcessor<Data1, Data2> {
@Override
public Data2 process(Data1 data1) throws Exception {
}
}
ItemWriter
public class MyItemWriter extends ItemStreamSupport implements ItemWriter<Data2> {
@Override
public void write(List<? extends Data2> listOfData) throws Exception {
}
}