AWS DMS CDC replication task failure error: load_common_wrapper( common_load_locker ) failed (at_common.c:69)

user_2878835 0
Viewed 211

We have an AWS DMS CDC replication task replicating data from an EC2 running MongoDB to Postgres. It has been running fine, DMS replication instance resources (RAM, CPU etc) all seem normal, however the task has started to fail, with:

Last Error The task stopped abnormally Stop Reason RECOVERABLE_ERROR Error Level RECOVERABLE

And error from Cloudwatch logs:

at_common_is_supported: load_common_wrapper( common_load_locker ) failed (at_common.c:69)

It is also possible to improve logging output further in the task settings JSON editor (additionally to the 5 fields that you can set in the console UI) by opening the task JSON editor and amending all the additional fields to: "Severity": "LOGGER_SEVERITY_DETAILED_DEBUG"

With the detailed_debug error logging enabled:

sqlite_execute_statement:361: Failed (at_sqlite.c:367)

sqlStatement=ALTER TABLE cdc_status ADD COLUMN read_volume_rollback INTEGER; (at_sqlite.c:325)

object already exists (at_sqlite.c:331)

Has anyone seen/ resolved this previously? Thanks in advance

aws-dms
1 Answers

In order to debug this you can try two things:

  1. If you have access to the mongod logs, you can check those. You will be able to see issues that DMS can not report back to you. The mongod logs are the system logs of the MongoDB process outputs.

  2. If you don't have access to the MongoDB logs, make sure that the user that you are using for the MongoDB source in DMS has the following roles:

First, you need to create a new role in the admin database:

use admin

db.createRole({
  role: "changestreamrole",
  privileges: [
    { resource: { db: "", collection: "" }, actions: ["find", "changeStream"] },
  ],
  roles: [{ role: "read", db: "local" }],
});

Second, you need to make sure that as an end result, the user you use needs to have the following roles:

[
  { role: "changestreamrole", db: "admin" },
  { role: "backup", db: "admin" },
  { role: "readWrite", db: "wur_university_rankings" },
]

readWrite and backup are MongoDB Built-in roles. The reason why we use backup is for the listDatabases privellage.

I hope this helps! Good luck!

user_34863040
Related