Every week I receive an updated statement and I have to save the data in the database. At the moment it has around 5000 records, but it's growing every week as new data is appended to the previous weekly statement. For each record, I need to: create if it doesn't exist, update if it's different, do nothing if unchanged. If I compare each record with existing records in the database, it becomes very slow. Another approach is to delete all the data, and create fresh records every time. But that also seems like a bad approach to me. What would be the proper way to handle this?

Create or update vs delete all records and create

Every week I receive an updated statement and I have to save the data in the database.

At the moment it has around 5000 records, but it's growing every week as new data is appended to the previous weekly statement.

For each record, I need to:

If I compare each record with existing records in the database, it becomes very slow.

Another approach is to delete all the data, and create fresh records every time. But that also seems like a bad approach to me.

What would be the proper way to handle this?