Sequelize.sync: Create indexes when table does not exist

Viewed 2665

According to documentation sequelize.sync() without {force: true} or {alter:true} is supposed to ignore already existing tables and create/sync only new ones. But there are at least two use cases when existing tables are not fully ignored and there are errors introduced.

My setup:

  1. sequelize.sync() is used as a pre-migration step to create tables that don't exist in the schema
  2. sequelize.migrate() is used to alter any existing tables.

Note: Sequelize models are treated as a single source of truth for reflecting database schema. They are always updated to reflect all the indexes/fields existing in the database.

Steps to reproduce

Step 1: Create User model with two fields name and email. Email has a unique index

const users = sequelizeClient.define('users', {
    name: {
      type: DataTypes.STRING,
      allowNull: false,
    },
    email: {
      type: DataTypes.STRING,
      allowNull: false,
    },
  }, {
    indexes: [
      {
        unique: true,
        fields: ['email'],
      },
    ],
  });

There is no migration, so the table is expected to be created using sequelize.sync(). Everything works as expected. Here are the generated SQL scripts.

Executing (default): CREATE TABLE IF NOT EXISTS "users" ("id"  SERIAL , "name" VARCHAR(255) NOT NULL, "email" VARCHAR(255) NOT NULL, PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'users' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): CREATE UNIQUE INDEX "users_email" ON "users" ("email")

Step 2 Add a new field phonenumber to the Users table and add a unique index. Add a migration that will alter the table structure and create the index. The sequelize.sync() is expected to ignore this table but migration is never executed as sequelize.sync() throws the following error.


Executing (default): CREATE TABLE IF NOT EXISTS "users" ("id"  SERIAL , "name" VARCHAR(255) NOT NULL, "email" VARCHAR(255) NOT NULL, PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'users' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): CREATE UNIQUE INDEX "users_phonenumber" ON "users" ("phonenumber")
{"_bitField":18087936,"_fulfillmentHandler0":{"name":"SequelizeDatabaseError","parent":{"name":"error","length":101,"severity":"ERROR","code":"42703","file":"indexcmds.c","line":"1083","routine":"ComputeIndexAttrs","sql":"CREATE UNIQUE INDEX \"users_phonenumber\" ON \"users\" (\"phonenumber\")"},"original":{"name":"error","length":101,"severity":"ERROR","code":"42703","file":"indexcmds.c","line":"1083","routine":"ComputeIndexAttrs","sql":"CREATE UNIQUE INDEX \"users_phonenumber\" ON \"users\" (\"phonenumber\")"},"sql":"CREATE UNIQUE INDEX \"users_phonenumber\" ON \"users\" (\"phonenumber\")"},"_trace":{"_promisesCreated":0,"_length":1},"level":"error","message":"Unhandled Rejection at: Promise "}

Here's the final model

const users = sequelizeClient.define('users', {
    name: {
      type: DataTypes.STRING,
      allowNull: false,
    },
    email: {
      type: DataTypes.STRING,
      allowNull: false,
    },
    phoneNumber: { // new field
      type: DataTypes.STRING,
      allowNull: false,
    },

  }, {
    indexes: [
      {
        unique: true,
        fields: ['email'],
      },
      { // new index
        unique: true,
        fields: ['phoneNumber'],
      },
    ],
  });

Can someone suggest a workaround here so the index creation happens only if the table doesn't exist


Another use case is when you add a new field with a comment

    fieldWithComment: {
      type: DataTypes.STRING,
      comment: 'my comment goes here',
    },

Generated SQL which obviously throws an error as the new column does not exist yet.

CREATE TABLE IF NOT EXISTS "users" (
    "id"   SERIAL, 
    "name" VARCHAR(255) NOT NULL, 
    "email" VARCHAR(255) NOT NULL, 
    "phonenumber" VARCHAR(255) NOT NULL, 
    "fieldWithComment" VARCHAR(255) , PRIMARY KEY ("id")); 
        COMMENT ON COLUMN "users"."fieldWithComment" IS 'my comment goes here';
2 Answers

The correct and non-destructive way to sync the database in production is via migrations. This way the order of the operations (new field creation, index creation etc.) is ensured.

So, generally, sync should only be used in development and testing environments.

Quoting from the official documentation:

sync({ force: true }) and sync({ alter: true }) can be destructive operations. Therefore, they are not recommended for production-level software. Instead, synchronization should be done with the advanced concept of Migrations, with the help of the Sequelize CLI.

More info here (Section: "Synchronization in production").

If anyone still struggling, not satisfied with the above answer because of sequelize updates, etc like me, here is how I solved the problem.

I was using underscore:true option.
So, I've changed the line fields: ["phone", "countryCode"], to fields: ["phone", "country_code"],.

sequelize.define(
  "user",
  {
    phone: {
      type: DataTypes.STRING(20),
      allowNull: false,
    },
    countryCode: {
      type: DataTypes.STRING(4),
      allowNull: false,
    },
    // other attributes ...
  },
  {
    freezeTableName: true,
    timestamps: true,
    underscored: true,
    indexes: [
      {
        unique: true,
        fields: ["phone", "country_code"],
      },
    ],
  }
);

Related