What are the best way to load incremental data into Database using Spark?

I have tried multiple ways to incremental load (upsert) into the Postgress Database (RDS) using Spark (with Glue Job) but did not find satisfactory performance.

I have tried the below ways:

1. Delete the records based on the primary Key and append new records

In this approach, I delete the records where the primary key exists in incremental data. This is done by Python API such as pg8000 etc. Once it gets successfully completed, I write data using Spark in append mode.

This approach works very smooth when the incremental load is less but with high volumes, Delete and writing takes time.

2. Writing data into a staging table and triggering DB store procedure

I have also tried this alternative way to handle the upsert with a relational database. Here I am writing data into a Staging table using Spark and triggering Store Procedure (with pg8000). The Store Procedure will upsert the data from the staging table to the final table.

With the approach, we have to manage multiple Store procedure and also have to depend on the Database (in case we are not owning that system). Handling the staging to final upsert will be tricky if the previous job gets failed after writing data into staging and before triggering the Store procedure. Overall I feel this approach needs good orchestration not only with the Spark but with Store procedure as well.

Is there any way that can handle upsert in a relational database (Postgres/MySQL/Oracle etc) very smoothly and efficiently using Spark?