NodeJS MongoDb - Export large data programmatically

Viewed 178

I have a Rest API.

One of my endpoints have to create a JSON Export from my database.

My database (~ 2M records).

Below my requests :

json.aggregate([
        {$sample: {size: limit}}, 
        {$project: {_id: 0, whenUpdated: 0}},
        {$match: {registered:true}}
    ]).toArray((error, collections) => {

I'm trying to create a random export without some field.

My limit could be something like 250k. Each record is a complex json.

My problem is request timeout. Something like 60s. I can't generate my export quickly enough.

I tryed mongoexport but i can't have random records.

Do you have an idea for export large random data in json or csv ?

Thanks

0 Answers
Related