PG::DependentObjectsStillExist error when trying to remove a uniqueness compound index

I was able to create a compound unique index using the following syntax:

add_index :my_table, [:col_1, :col_2], unique: true, algorithm: :concurrently, name: :my_unique_compound_index

However, when I try to remove that index using remove_index instead of add_index, Postgresql raises the following error:

PG::DependentObjectsStillExist: ERROR: cannot drop index my_unique_compound_index because constraint my_unique_compound_index on table my_table requires it.

HINT: You can drop constraint my_unique_compound_index on table my_table instead.

I've tried removing the index with and without the concurrent algorithm (which I learnt about in How to Create Postgres Indexes Concurrently in ActiveRecord Migrations) but that doesn't change the output.
I've tried removing the index inside a change_table block but it gives the exact same error.

Am I doing it wrong, or is their something particular to be done to force Postgresql to drop a unique compound index? Not the most helpful hint by the way :-)

Edit for additional debug information:

I'm using Rails 6.1.5 with PostgreSQL 12.9.

Here's the output when running \d+ my_table in psql:

Indexes:
    "my_table_pkey" PRIMARY KEY, btree (id)
    "my_unique_compound_index" UNIQUE CONSTRAINT, btree (col_1, col_2)
    "index_my_table_on_another_id" btree (other_id)
Foreign-key constraints:
    "fk_rails_4eb0b2d2d4" FOREIGN KEY (other_table_id) REFERENCES other_tables(id)
    "fk_rails_f7374ea9ef" FOREIGN KEY (yet_another_table_id) REFERENCES yet_another_tables(id)
Access method: heap