I have an existing system that works in the following way.
- System One: This is a Spring Boot Rest API Application that Pulls/Download documents (PDF) from S3, converts them to base64, and sends them to System Two
- System Two: This accepts documents in base64 format from System one and inserts them into its DB
Problem:
- When there are multiple documents of big size, we always face a Heap Memory Issue in System One
To overcome this we came up with the approach that we will send the Documents in a sequential manner one by one to System Two from System One. But I doubt at some point when multiple users will try to perform activity on the Application, System One might again face Memory Issue if the GC does not execute.
Question:
- Is there a way to directly Stream/Send S3 Object (PDF) files to an API endpoint without downloading them on System One and converting them to base64 while Streaming, since System Two only accepts Documents in this (base64) form only