Cloud firestore bandwidth exhausted error

user_12959708 0
Viewed 2693

We are using cloud firestore as our database and getting following error when rate of parallel reads from database increases.

details:  "Bandwidth exhausted"    
message:  "8 RESOURCE_EXHAUSTED: Bandwidth exhausted"    
stack:  "Error: 8 RESOURCE_EXHAUSTED: Bandwidth exhausted
at callErrorFromStatus (/usr/service/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
at Http2CallStream.call.on (/usr/service/node_modules/@grpc/grpc-js/build/src/call.js:79:34)
at Http2CallStream.emit (events.js:198:15)
at process.nextTick (/usr/service/node_modules/@grpc/grpc-js/build/src/call-stream.js:100:22)
at processTicksAndRejections (internal/process/task_queues.js:79:9)"

We couldn't find what is the rate limits. Could you please let me know what are the read rate limits and in which cases firestore returns Bandwidth exhausted error?

Note: Billing is enabled in our project. The problem is we can't find what limit we are reaching.

google-cloud-firestore
2 Answers

The RESOURCE_EXHAUSTED error indicates that the project exceeded either its quota or the region/multi-region capacity, so probably your app is doing more reads than expected given what you described. You can check more details in this documentation.

You can check the free quotas and the standard limits on this link and the pricing for what exceeds those numbers on this link. It's important to note that, if you choose to allow your app to go further than the free quotas, you must enable billing for your Cloud Platform project, here is a how to.

You can also check how much your app is actually using of the quotas on app engine on the section below:

enter image description here

Hope This helps.

user_128577030

If your are reading all data from Firebase, this issue happens to you, I had the same problem for reading all data from firebase, after a while, I figured out if we stop this process and run a new process we can pass the error and continue the job.

then I used child-process and it helped me:

  • I did write a parent script and a child script,
  • Parent script runs child script as a child-process,
  • The child goes through a collection until get [8 RESOURCE_EXHAUSTED] error, then send a message to parent to inform it from the error.
  • Then parent kills child and create a new one and tells it where to start reading again.

This is a solution that works 100 percent, but it's a little advance and beginners-intermediates may could not able to implement it.

update: I have written a complete instruction on GitHub Gist for this issue, you can check it: https://gist.github.com/navidshad/973e9c594a63838d1ebb8f2c2495cf87

user_39690060
Related