Idempotency and foreign key constraints when using Postgres pg_dump files with Flyway

user_3311675 0
Viewed 45

I am trying to use some pg_dump generated migration scripts with Flyway. The first migration script is for schema only. The other migration scripts load seed data into various tables using the Postgres COPY command. These seed-data scripts are going to exist as Flyway repeatable migration scripts. This setup presents two issues.

  • When Flyway loads the seed data from the migration scripts, I'm getting foreign key constraint violations since I don't have the various tables being seeded in the correct order. There are a large number of tables to deal with, so is there an easy way to work around this so that I don't have to try to reorder my COPY's?
  • Since the seed data is going to be in repeatable migration scripts, these need to be idempotent. Is there a way to do this with the Postgres COPY command? I'm trying to avoid having to convert this to INSERTs since it will hurt performance and also make my migrations files huge.
flyway postgresql
1 Answers

The trick here for idempotency is to delete the data from the files in the correct dependency order, and when you've done that, to likewise load the data in the correct dependency order. The correct dependency order for deleting the data is worked out by obtaining the target tables for every foreign key constraint and ensuring that no data from a table is ever deleted when it is the target of a table whose data is yet to be deleted. This list of tables in dependency order is usually called a 'manifest' and is required also for CREATE statements and for the PgSQL COPY. The Public domain PowerShell-based Flyway Teamworks framework will create the manifest for you.

user_128397940
Related