I have 10 unique id and using that Id I fetch data from the database and I do a rest call with those data and I get an async response as pending as it takes some time to sync with the third party service, as soon as I receive the response from the API I update my data's(retrieved from DB) to pending?
our solution:
*
the Initial idea was to have different Kubernetes pods for each unique ID by this each can scale independently the problem is that there could be pods that can be ideal and would be a waste of resources, we are trying to address this case
Code flow:
- we read the data in batch from mysqlDB with each ID
- after reading it we then update those columns with status as pending so that on the next polling it doesn’t read the same data.
- we then push the data to downstream and update the status as finished
is there any design pattern that I should look for or any suggestions that would be a help