My organization moves data for customers between systems, these integrations are in BizTalk and are done by file, sometimes to/from APIs. More and more customers are switching to APIs so we are facing more and more API to API integrations.
I'm mostly a backend developer but have been tasked with finding out how we can find a more generic pattern or system to make these integrations, we are talking close to a thousand of integrations. But not thousands of different APIs, many customers use the same sort of systems.
What I want is a solution that:
- Fetches data from the source api
- Transforms the data to the format for the target api
- Sends the data to the target api
Another requirement is that it should be possible to set a schedule when these jobs should run. This is easily done in BizTalk but as mentioned there will be thousands of integrations and if we need to change something in one of the steps it will be a lot of work.
My vision is something that holds interfaces to all APIs that we communicate with and also contains the scheduled jobs we want to be run between them. Preferrably with logging/tracking.
There must be something out there that does this? Suggestions?
NOTE: No cloud-based solutions since they are not allowed in our organization.