How do I solve this error of Pymongo? cursor id not found pymongo

user_13823863 0
Viewed 2515

I'm getting the whole data of some collection in MongoDB and after a while (like 30 or 60 minutes), the script raises the following error:

pymongo.errors.CursorNotFound: cursor id 1801580172063793986 not found, full error: {'ok': 0.0, 'errmsg': 'cursor id 1801580172063793986 not found', 'code': 43, 'codeName': 'CursorNotFound'}

This error occurs after the 24k documents. I'm using Django and Pymongo connected to the database in the local server. The collection has like 60k documents.

This is how I'm getting the data:

client = MongoClient(settings.MONGO_HOST, settings.MONGO_PORT)
collection = client[settings.MONGO_DB].collection
cursor = collection.find(no_cursor_timeout=True)
for document in cursor:
    # getting the data from the user

Just in case, I'm using:

  • Python 3.8
  • Django 3.1.4
  • Pymongo 3.11.0
  • Mongod 4.4.2 (for the local server)
  • Ubuntu 20.04
pymongo mongodb ubuntu django python
3 Answers

This is not the solution to the error, but this is a way to avoid the error. In order to use the cursor the minimum time possible, you can just save all the data in a list or something like that.

client = MongoClient(settings.MONGO_HOST, settings.MONGO_PORT)
collection = client[settings.MONGO_DB].collection
cursor = collection.find(no_cursor_timeout=True)

collection_data = [document for document in cursor]
for document in collection_data:
    # using the data
user_138238630
user_95409250

Try setting the cursor timeout globally via cursorTimeoutMillis parameter.

In your terminal, type:

$ mongod --setParameter cursorTimeoutMillis=600000
user_69523590
Related