Java Spring Mongo DB Bulk Upsert Performance

Viewed 138

I am trying to upsert a POJO list about 50000 items using bulk operation api in Spring.

I am using following code block to upsert the records, however it is taking about 15 minutes to update the whole collection.

public void saveHistory(List<History> history) {

        List<Pair<Query, Update>> updates = new ArrayList<>(history.size());
        history.forEach(item -> {
            Query query = Query.query(Criteria.where("Id").is(item.getUserId()));
            Update update = new Update();
            update.set("_id", item.getUserId());
            update.set(“auditHistoryMap", item.getAuditHistoryMap());
            updates.add(Pair.of(query, update));
        });

        BulkOperations bulkOperations = mongoTemplate.bulkOps(BulkOperations.BulkMode.UNORDERED, Historyclass);
        bulkOperations.upsert(updates);
        bulkOperations.execute();
    }

However if the POJO were converted to Document type prior to upsert whole collection can be upserted in about 3 minutes .

public <T> void bulkUpdate(String collectionName, List<T> documents, Class<T> tClass) {
        BulkOperations bulkOps = mongoTemplate.bulkOps(BulkOperations.BulkMode.UNORDERED, tClass, collectionName);
        for (T document : documents) {
            Document doc = new Document();
            mongoTemplate.getConverter().write(document, doc);
            Query query = new Query(Criteria.where("_id").is(doc.get("_id")));
            Document updateDoc = new Document();
            updateDoc.append("$set", doc);
            Update update = Update.fromDocument(updateDoc, "_id");
            bulkOps.upsert(query, update);
        }
        bulkOps.execute();
    }

What is the reason for such disparity ?

0 Answers
Related