mongodb unable establish remote cursors

Viewed 174

Anybody seen this messages before in mongodb sharded cluster 4.0.16 mongos during balancing:

   2021-10-24T13:26:03.723+0200 I QUERY    [conn636] Unable to establish remote cursors - {error: StaleConfig{ ns: "prod.assests", vReceived: Timestamp(1841827, 4), vReceivedEpoch: ObjectId('584af7e7cec9edde8118d5ac'), vWanted: Timestamp(1841826, 1), vWantedEpoch: ObjectId('584af7e7cec9edde8118d5ac') }: version mismatch detected for prod.assests, numActiveRemotes: 0}
   2021-10-24T13:26:03.723+0200 I QUERY    [conn768] Unable to establish remote cursors - {error: StaleConfig{ ns: "prod.assests", vReceived: Timestamp(1841827, 4), vReceivedEpoch: ObjectId('584af7e7cec9edde8118d5ac'), vWanted: Timestamp(1841826, 1), vWantedEpoch: ObjectId('584af7e7cec9edde8118d5ac') }: version mismatch detected for prod.assests, numActiveRemotes: 0}
   2021-10-24T13:26:03.723+0200 I QUERY    [conn507] Unable to establish remote cursors - {error: StaleConfig{ ns: "prod.assests", vReceived: Timestamp(1841827, 4), vReceivedEpoch: ObjectId('584af7e7cec9edde8118d5ac'), vWanted: Timestamp(1841826, 1), vWantedEpoch: ObjectId('584af7e7cec9edde8118d5ac') }: version mismatch detected for prod.assests, numActiveRemotes: 4}

This messages appear only during balancing in a 5x shards (1TB on ssd each) cluster in all mongoses... Also messages appear only during the mentioned collection balancing , other collections balance with no such messages in same cluster...

The questions:

1.Any advice will be highly appreciated on why this messages appear?

2.What is the meaning of this message, do I need to worry about ?

  1. Is there possible fix?

Balancing seems working correctly ...

Customers not complaining , queries to cluster working correctly ...

RangeDeleter working correctly ....

There is no network or storage resources issues ...

The load is minimal during the time of balancing ...

1 Answers
  1. This message is expected behaviour during balancing when there is read request for documents already migrated to other shard.
  2. The meaning is that the mongos is not able to establish remote cursor to the old shard since the config is reported stale and data is moved to the new shard.
  3. No fix is necessary this is informative message only.
Related