Should I be removing tombstones from my Cloudant database?

Viewed 35

I have a Cloudant database with 4 million documents and 27 million deleted documents ("tombstones"). Is having so many tombstones a problem and, if so, how do I get rid of them?

1 Answers

"Tombstones" occupy space and so contribute to your bill. They also increase the time for new replications to complete or new indexes to build. So in general it is good practice to periodically remove these tombstones. The best way to do it is to replicate your database with a filter that leaves deleted documents behind.

Replications are started by creating a document in the _replicator database like so:

{
  "_id": "myfirstreplication",
  "source" : "http://<username1>:<password1>@<account1>.cloudant.com/<sourcedb>",
  "target" : "http://<username2:<password2>@<account2>.cloudant.com/<targetdb>",
  "selector": {
    "_deleted": {
      "$exists": false
    }
  }
}

where the source is the original database and the target is the new empty database. The selector is the filter that checks each document before replicating - in this case we only want documents without a deleted attribute (a document that hasn’t been deleted).

This replication will result in a brand new database with no tombstones. Point your application to this new database and then delete the old one with the tombstones.

In this blog post there are other, more complex, scenarios that you may want to consider.

Related