I have setup an ingest pipeline for streaming json data which gets ingested into AWS S3 data lake through pyspark and also Oracle db.
In Oracle db, primary keys are created using sequences. What is the most efficient way I can design the data lake ingestion to have the primary keys always in-sync with the oracle db?
Both the ingestions happen parallely.