Mongoose: find document and trim array elements that don't match with the condition

user_16580493 0
Viewed 112

i have documents like this :

{
  "_id": {
    "$oid": "630656bfc64b4c94c5c5a614"
  },
  "collectionId": {
    "$oid": "630654a37a2a8ea9e11b6803"
  },
  "__v": 0,
  "createdAt": {
    "$date": {
      "$numberLong": "1661359804637"
    }
  },
  "stats": [
    {
      "floor": 0.005,
      "volume": 3.288569999999999,
      "listedCount": 57,
      "timestamp": {
        "$date": {
          "$numberLong": "1661359800000"
        }
      },
      "_id": {
        "$oid": "630656bc7a2a8ea9e11bc74b"
      }
    },
    {
      "floor": 0.0001,
      "volume": 3.081616000000002,
      "listedCount": 88,
      "timestamp": {
        "$date": {
          "$numberLong": "1661360400000"
        }
      },
      "_id": {
        "$oid": "6306591e7a2a8ea9e11c2479"
      }
    },
  ],
  "updatedAt": {
    "$date": {
      "$numberLong": "1661530019840"
    }
  }
}

Where stats is array of object and sometimes i can have 1000+ elements in that array. As you can see objects inside stats have a timestamp. Now i want to only return documents matching a specific condition like so :

{ 'stats.timestamp': { '$gt': '2022-08-22' } }

For now it's fine this will return all the documents that have stats.timestamp > 2022-08-22

But as i said my array of stat can be 1000+ elements and in the future it will grow more than 10000+

I was wondering if there is a way to trim array of stats and delete all the elements that don't match the condition.

{ 'stats.timestamp': { '$gt': '2022-08-22' } }

I want that the elements inside stats are only the one that have timestamp > 2022-08-22.

i want something without aggregate.

mongoose node.js mongodb
3 Answers

find document and trim array elements that don't match with the condition

You can use the find method with a projection to trim the array based upon the condition. This feature requires MongoDB v4.4. The following query runs from mongosh (or mongo shell) using the data from the post.

db.collection.find(
  { }, 
  { 
      data: { 
          $filter: { 
              input: "$data", 
              cond: { 
                  $gt: [ 
                      { $dateToString: { 
                          format: "%Y-%m-%d", 
                          date: { $toDate: "$$this.timestamp" } 
                      } }, 
                      "2022-08-24"     // <----- the date criteria in YYYY-mm-dd format
                  ] 
              } 
          } 
      } 
  }
)
user_46793200

You can use a nodejs function that executes once every 24 hours and trims the array like -

async function trimUnnecessaryItems(){
await db.model.updateMany({},{$pull: {stats: {timestamp: {$gt': '2022-08-22}}}})
}
user_122720460

I think for the update and trimming, you will need to use update with $filter in aggregate. I am not sure if there is a way that allow you to skip an aggregation without resorting to js level / other application/script level's help.

db.collection.update({
  "stats.timestamp": {
    "$lte": ISODate("2022-08-22")
  }
},
[
  {
    $set: {
      "stats": {
        $filter: {
          input: "$stats",
          as: "s",
          cond: {
            $gt: [
              "$$s.timestamp",
              ISODate("2022-08-22")
            ]
          }
        }
      }
    }
  }
],
{
  multi: true
})

Here is the Mongo playground for your reference.

user_147326690
Related