Mongo bulk filter and add concatenated field

Question

I have a Mongo collection that I'm trying to update (in PyMongo). I have documents like {'state': 'ny', 'city': 'nyc', 'race': 'b', 'ethnicity': 'h'} and I'm trying to bulk operate on these documents by matching certain criteria and created a concatenated 'race_ethnicity' field.

A pseudo example might be:

filter = {
    'state': 'ny',
    'city': 'nyc',
    'race': {"$exists": True},
    'ethnicity': {"$exists": True},
    'race_ethnicity': {"$exists": False}
}

action = {'$addFields': {'race_ethnicity': {'$concat': ['$race', '$ethnicity']}}}))

Using the above document, the updated document would be: {'state': 'ny', 'city': 'nyc', 'race': 'b', 'ethnicity': 'h', 'race_ethnicity': 'bh'}.

I want to bulk match, and bulk update a collection like this — how do I go about this without getting a BulkWriteError?

*** What I tried:

updates = []
updates.append(UpdateMany({
        'state': 'ny',
        'city': 'nyc',
        "race": {"$exists": True},
        "ethnicity": {"$exists": True},
        "race_ethnicity": {"$exists": False}
    },
    {'$addFields': {'race_ethnicity': {'$concat': ['$race', '$ethnicity']}}}))
self.db.bulk_write(updates)

This produced the following error:

pymongo.errors.BulkWriteError: batch op errors occurred

user_8987128 · Answer

Your bulk write payload is not correct as per bulkWrite() syntax,
$addFields is an aggregation pipeline stage you can not use in regular update queries, so to resolve this You can use update with aggregation pipeline starting from MongoDB 4.2,

updates = []

updates.append({
  'updateMany': {
    'filter': {
      'state': 'ny',
      'city': 'nyc',
      'race': { '$exists': True},
      'ethnicity': { '$exists': True},
      'race_ethnicity': { '$exists': False}
    },
    'update': [
      {
        '$set': { 'race_ethnicity': { '$concat': ['$race', '$ethnicity'] } }
      }
    ]
  }
})

self.db.bulk_write(updates)

Playground

Mongo bulk filter and add concatenated field

1 Answers