MongoDB report an error while using `db.TestCollection.find({}).count()` to query the count of Collection

user_15691082 0
Viewed 37
TestUser> db.TestCollection.find({}).count()
[2021-06-01 11:02:58] com.mongodb.MongoSocketReadException: Prematurely reached end of str

I have Five hundred million documents in TestCollection,MongoDB report an error while i was trying to get the count of it .What can i do to avoid that? I have tried resetting the max_connection_idle_time to 6000ms , but it didnt work.

Thanks!

mongodb
1 Answers

Your MongoDB Version : 4.0

To start with the default count method, cursor count is getting deprecated since 4.0 due to some consistency issues.

Now the issues, the new methods for the count are broken into two parts

  1. estimatedDocumentCount()
  2. countDocuments()

The first one is used to get count of documents for the collection, it directly gives you the data from the metadata of the collection hence can be slightly delayed if your collection has a lot of writes taking place on it.

The second one entertains the query filter part but under the hood works using this method:

db.collection.aggregate([
   { $match: <query> },
   { $group: { _id: null, n: { $sum: 1 } } }
])

The problem with this approach is for the DBMS to get you the count it has to manually count those documents, for which it will have to keep those documents in the WiredTiger Internal Cache, and hence on a big collection it would be slow and problematic, directly proportional to size of the collection.

For such scenarios try not repeating the count, use computed pattern discussed here.

There is a good possibility that in the current query you are running it is doing a manual scan and is struggling in getting all the data in WT Cache if it wasn't already there. For starters if you have to use query predicate and count then make sure there is an index over the search term but my vote, if possible, will always go to using computed pattern.

user_52253630
Related