How to copy a collection from one database to another in MongoDB

Viewed 237273

Is there a simple way to do this?

23 Answers

for huge size collections, you can use Bulk.insert()

var bulk = db.getSiblingDB(dbName)[targetCollectionName].initializeUnorderedBulkOp();
db.getCollection(sourceCollectionName).find().forEach(function (d) {
    bulk.insert(d);
});
bulk.execute();

This will save a lot of time. In my case, I'm copying collection with 1219 documents: iter vs Bulk (67 secs vs 3 secs)

Unbelievable how many up-votes are given for agonizingly slow one-by-one copy of data.

As given in other answers the fastest solution should be mongodump / mongorestore. There is no need to save the dump to your local disk, you can pipe the dump directly into mongorestore:

mongodump --db=some_database --collection=some_collection --archive=- | mongorestore --nsFrom="some_database.some_collection" --nsTo="some_or_other_database.some_or_other_collection" --archive=-

In case you run a sharded cluster, the new collection is not sharded by default. All data is written initially to your primary shard. This may cause problems with disk space and put additional load to your cluster for balancing. Better pre-split your collection like this before you import the data:

sh.shardCollection("same_or_other_database.same_or_other_collection", { <shard_key>: 1 });
db.getSiblingDB("config").getCollection("chunks").aggregate([
   { $match: { ns: "some_database.some_collection"} },
   { $sort: { min: 1 } },
   { $skip: 1 }
], { allowDiskUse: true }).forEach(function (chunk) {
   sh.splitAt("same_or_other_database.same_or_other_collection", chunk.min)
})

There are different ways to do the collection copy. Note the copy can happen in the same database, different database, sharded database or mongod instances. Some of the tools can be efficient for large sized collection copying.

Aggregation with $merge: Writes the results of the aggregation pipeline to a specified collection. Note that the copy can happen across databases, even the sharded collections. Creates a new one or replaces an existing collection. New in version 4.2. Example: db.test.aggregate([ { $merge: { db: "newdb", coll: "newcoll" }} ])

Aggregation with $out: Writes the results of the aggregation pipeline to a specified collection. Note that the copy can happen within the same database only. Creates a new one or replaces an existing collection. Example: db.test.aggregate([ { $out: "newcoll" } ])

mongoexport and mongoimport: These are command-line tools. mongoexport produces a JSON or CSV export of collection data. The output from the export is used as the source for the destination collection using the mongoimport.

mongodump and mongorestore: These are command-line tools. mongodump utility is for creating a binary export of the contents of a database or a collection. The mongorestore program loads data from a binary database dump created by mongodump into the destination.

db.cloneCollection(): Copies a collection from a remote mongod instance to the current mongod instance. Deprecated since version 4.2.

db.collection.copyTo(): Copies all documents from collection into new a Collection (within the same database). Deprecated since version 3.0. Starting in version 4.2, MongoDB this command is not valid.

NOTE: Unless said the above commands run from mongo shell.

Reference: The MongoDB Manual.

You can also use a favorite programming language (e.g., Java) or environment (e.g., NodeJS) using appropriate driver software to write a program to perform the copy - this might involve using find and insert operations or another method. This find-insert can be performed from the mongo shell too.

You can also do the collection copy using GUI programs like MongoDB Compass.

If RAM is not an issue using insertMany is way faster than forEach loop.

var db1 = connect('<ip_1>:<port_1>/<db_name_1>')
var db2 = connect('<ip_2>:<port_2>/<db_name_2>')

var _list = db1.getCollection('collection_to_copy_from').find({})
db2.collection_to_copy_to.insertMany(_list.toArray())

Many right answers here. I would go for mongodump and mongorestore in a piped fashion for a large collection:

mongodump --db fromDB --gzip --archive | mongorestore --drop --gzip --archive --nsFrom "fromDB.collectionName" --nsTo "toDB.collectionName"

although if I want to do quick copy, its slow but it works:

use fromDB 
db.collectionName.find().forEach(function(x){
   db.getSiblingDB('toDB')['collectionName'].insert(x);
});"

use "Studio3T for MongoDB" that have Export and Import tools by click on database , collections or specific collection download link : https://studio3t.com/download/

To copy a collection (myCollection1) from one database to another in MongoDB,

**Server1:**
myHost1.com 
myDbUser1
myDbPasword1
myDb1
myCollection1

outputfile:
myfile.json 

**Server2:**
myHost2.com 
myDbUser2
myDbPasword2
myDb2
myCollection2 

you can do this:

mongoexport  --host myHost1.com --db myDb1 -u myDbUser1  -p myDbPasword1 --collection myCollection1   --out  myfile.json 

then:

mongoimport  --host myHost2.com --db myDb2 -u myDbUser2  -p myDbPasword2 --collection myCollection2   --file myfile.json 

Another case , using CSV file:

Server1:
myHost1.com 
myDbUser1
myDbPasword1
myDb1
myCollection1
fields.txt
    fieldName1
    fieldName2

outputfile:
myfile.csv

Server2:
myHost2.com 
myDbUser2
myDbPasword2
myDb2
myCollection2

you can do this:

mongoexport  --host myHost1.com --db myDb1 -u myDbUser1  -p myDbPasword1 --collection myCollection1   --out  myfile.csv --type=csv

add clolumn types in csv file (name1.decimal(),name1.string()..) and then:

mongoimport  --host myHost2.com --db myDb2 -u myDbUser2  -p myDbPasword2 --collection myCollection2   --file myfile.csv --type csv --headerline --columnsHaveTypes

The simplest way to import data from the existing MongoDB atlas cluster DB is using mongodump & mongorestore commands.

To create the dump from existing DB you can use:

mongodump --uri="<connection-uri>"

There are other options for connection which can be lookup here: https://www.mongodb.com/docs/database-tools/mongodump/

After the dump is successfully created in a dump/ directory, you can use import that data inside your other db like so:

mongorestore --uri="<connection-uri-of-other-db>" <dump-file-location>

Similarly for mongorestore, there are other connection options that can be looked up along with commands to restore specific collections: https://www.mongodb.com/docs/database-tools/mongorestore/

The dump file location will be inside the dump directory. There may be a subdirectory with the same name as DB name which you dumped. For example if you dumped test DB, then dump file location would be /dump/test

Related