MongoDB: Searching a text field using mathematical operators

Viewed 312

I have documents in a MongoDB as below -

[
{
    "_id": "17tegruebfjt73efdci342132",
    "name": "Test User1",
    "obj":  "health=8,type=warrior",
},
{
    "_id": "wefewfefh32j3h42kvci342132",
    "name": "Test User2",
    "obj":  "health=6,type=magician",
}
.
.
]

I want to run a query say health>6 and it should return the "Test User1" entry. The obj key is indexed as a text field so I can do {$text:{$search:"health=8"}} to get an exact match but I am trying to incorporate mathematical operators into the search.

I am aware of the $gt and $lt operators, however, it cannot be used in this case as health is not a key of the document. The easiest way out is to make health a key of the document for sure, but I cannot change the document structure due to certain constraints.

Is there anyway this can be achieved? I am aware that mongo supports running javascript code, not sure if that can help in this case.

2 Answers

I don't think it's possible in $text search index, but you can transform your object conditions to an array of objects using an aggregation query,

  • $split to split obj by "," and it will return an array
  • $map to iterate loop of the above split result array
  • $split to split current condition by "=" and it will return an array
  • $let to declare the variable cond to store the result of the above split result
  • $first to return the first element from the above split result in k as a key of condition
  • $last to return the last element from the above split result in v as a value of the condition
  • now we have ready an array of objects of string conditions:
  "objTransform": [
    { "k": "health", "v": "9" },
    { "k": "type", "v": "warrior" }
  ]
  • $match condition for key and value to match in the same object using $elemMatch
  • $unset to remove transform array objTransform, because it's not needed
db.collection.aggregate([
  {
    $addFields: {
      objTransform: {
        $map: {
          input: { $split: ["$obj", ","] },
          in: {
            $let: {
              vars: {
                cond: { $split: ["$$this", "="] }
              },
              in: {
                k: { $first: "$$cond" },
                v: { $last: "$$cond" }
              }
            }
          }
        }
      }
    }
  },
  {
    $match: {
      objTransform: {
        $elemMatch: {
          k: "health",
          v: { $gt: "8" }
        }
      }
    }
  },
  { $unset: "objTransform" }
])

Playground


The second upgraded version of the above aggregation query to do less operation in condition transformation if it's possible to manage in your client-side,

  • $split to split obj by "," and it will return an array
  • $map to iterate loop of the above split result array
  • $split to split current condition by "=" and it will return an array
  • now we have ready a nested array of string conditions:
  "objTransform": [
    ["type", "warrior"],
    ["health", "9"]
  ]
  • $match condition for key and value to match in the array element using $elemMatch, "0" to match the first position of the array and "1" to match the second position of the array
  • $unset to remove transform array objTransform, because it's not needed
db.collection.aggregate([
  {
    $addFields: {
      objTransform: {
        $map: {
          input: { $split: ["$obj", ","] },
          in: { $split: ["$$this", "="] }
        }
      }
    }
  },
  {
    $match: {
      objTransform: {
        $elemMatch: {
          "0": "health",
          "1": { $gt: "8" }
        }
      }
    }
  },
  { $unset: "objTransform" }
])

Playground

Using JavaScript is one way of doing what you want. Below is a find that uses the index on obj by finding documents that have health= text followed by an integer (if you want, you can anchor that with ^ in the regex).

It then uses a JavaScript function to parse out the actual integer after substringing your way past the health= part, doing a parseInt to get the int, and then the comparison operator/value you mentioned in the question.

db.collection.find({
    // use the index on obj to potentially speed up the query
    "obj":/health=\d+/,
    // now apply a function to narrow down and do the math
    $where: function() {
        var i = this.obj.indexOf("health=") + 7;
        var s = this.obj.substring(i);
        var m = s.match(/\d+/);
        
        if (m)
            return parseInt(m[0]) > 6;       
        return false;
    }
})

You can of course tweak it to your heart's content to use other operators.

NOTE: I'm using the JavaScript regex capability, which may not be supported by MongoDB. I used Mongo-Shell r4.2.6 where it is supported. If that's the case, in the JavaScript, you will have to extract the integer out a different way.

I provided a Mongo Playground to try it out in if you want to tweak it, but you'll get

Invalid query:

Line 3: Javascript regex are not supported. Use "$regex" instead

until you change it to account for the regex issue noted above. Still, if you're using the latest and greatest, this shouldn't be a limitation.

Performance

Disclaimer: This analysis is not rigorous.

I ran two queries against a small collection (a bigger one could possibly have resulted in different results) with Explain Plan in MongoDB Compass. The first query is the one above; the second is the same query, but with the obj filter removed.

enter image description here

and

enter image description here

As you can see the plans are different. The number of documents examined is fewer for the first query, and the first query uses the index.

The execution times are meaningless because the collection is small. The results do seem to square with the documentation, but the documentation seems a little at odds with itself. Here are two excerpts

Use the $where operator to pass either a string containing a JavaScript expression or a full JavaScript function to the query system. The $where provides greater flexibility, but requires that the database processes the JavaScript expression or function for each document in the collection.

and

Using normal non-$where query statements provides the following performance advantages:

  • MongoDB will evaluate non-$where components of query before $where statements. If the non-$where statements match no documents, MongoDB will not perform any query evaluation using $where.
  • The non-$where query statements may use an index.

I'm not totally sure what to make of this, TBH. As a general solution it might be useful because it seems you could generate queries that can handle all of your operators.

Related