How to Perform Large Deletes in the Firebase Realtime Database

user_5557591 0
Viewed 205

I am using Firebase realtime database. I want to delete a node which has huge records. So i can not delete with normal ways. When i try to delete with using rest, i can not do that. Default setting is "large". I have to set defaultWriteSizeLimit to "unlimited".

$firebase database:settings:set defaultWriteSizeLimit unlimited --instance <dbname>

When i execute above code, i receive an error message.

"Error: Unexpected error fetching configs at defaultWriteSizeLimit"

Below is firebase-debug.log:

[debug] [2020-12-09T09:24:13.317Z] ----------------------------------------------------------------------
[debug] [2020-12-09T09:24:13.318Z] Command:       /usr/local/lib/node_modules/firebase-tools/lib/bin/firebase.js /Users/emrekhan/.cache/firebase/tools/lib/node_modules/firebase-tools/lib/bin/firebase database:settings:set defaultWriteSizeLimit unlimited --instance dbname
[debug] [2020-12-09T09:24:13.318Z] CLI Version:   8.17.0
[debug] [2020-12-09T09:24:13.318Z] Platform:      darwin
[debug] [2020-12-09T09:24:13.318Z] Node Version:  v12.18.1
[debug] [2020-12-09T09:24:13.319Z] Time:          Wed Dec 09 2020 12:24:13 GMT+0300 (GMT+03:00)
[debug] [2020-12-09T09:24:13.319Z] ----------------------------------------------------------------------
[debug] [2020-12-09T09:24:13.319Z] 
[debug] [2020-12-09T09:24:13.322Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[debug] [2020-12-09T09:24:13.323Z] > authorizing via signed-in user
[debug] [2020-12-09T09:24:13.324Z] [iam] checking project dbname for permissions ["firebase.projects.get","firebasedatabase.instances.update"]
[debug] [2020-12-09T09:24:13.325Z] > refreshing access token with scopes: ["email","https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","openid"]
[debug] [2020-12-09T09:24:13.325Z] >>> HTTP REQUEST POST https://www.googleapis.com/oauth2/v3/token  
 <request body omitted>
[debug] [2020-12-09T09:24:13.668Z] <<< HTTP RESPONSE 200 {"date":"Wed, 09 Dec 2020 09:24:14 GMT","expires":"Mon, 01 Jan 1990 00:00:00 GMT","pragma":"no-cache","cache-control":"no-cache, no-store, max-age=0, must-revalidate","content-type":"application/json; charset=utf-8","vary":"X-Origin, Referer, Origin,Accept-Encoding","server":"scaffolding on HTTPServer2","x-xss-protection":"0","x-frame-options":"SAMEORIGIN","x-content-type-options":"nosniff","alt-svc":"h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","accept-ranges":"none","transfer-encoding":"chunked"}
[debug] [2020-12-09T09:24:13.682Z] >>> HTTP REQUEST POST https://cloudresourcemanager.googleapis.com/v1/projects/dbname:testIamPermissions  
 {"permissions":["firebase.projects.get","firebasedatabase.instances.update"]}
[debug] [2020-12-09T09:24:14.782Z] <<< HTTP RESPONSE 200 {"content-type":"application/json; charset=UTF-8","vary":"X-Origin, Referer, Origin,Accept-Encoding","date":"Wed, 09 Dec 2020 09:24:15 GMT","server":"ESF","cache-control":"private","x-xss-protection":"0","x-frame-options":"SAMEORIGIN","x-content-type-options":"nosniff","server-timing":"gfet4t7; dur=768","alt-svc":"h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","accept-ranges":"none","transfer-encoding":"chunked"}
[debug] [2020-12-09T09:24:14.783Z] >>> HTTP REQUEST GET https://firebasedatabase.googleapis.com/v1beta/projects/dbname/locations/-/instances/dbname
 
[debug] [2020-12-09T09:24:15.499Z] <<< HTTP RESPONSE 200 {"content-type":"application/json; charset=UTF-8","vary":"X-Origin, Referer, Origin,Accept-Encoding","date":"Wed, 09 Dec 2020 09:24:16 GMT","server":"ESF","cache-control":"private","x-xss-protection":"0","x-frame-options":"SAMEORIGIN","x-content-type-options":"nosniff","alt-svc":"h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","accept-ranges":"none","transfer-encoding":"chunked"}
[debug] [2020-12-09T09:24:15.501Z] > refreshing access token with scopes: []
[debug] [2020-12-09T09:24:15.501Z] >>> HTTP REQUEST POST https://www.googleapis.com/oauth2/v3/token  
 <request body omitted>
[debug] [2020-12-09T09:24:15.766Z] <<< HTTP RESPONSE 200 {"date":"Wed, 09 Dec 2020 09:24:16 GMT","cache-control":"no-cache, no-store, max-age=0, must-revalidate","expires":"Mon, 01 Jan 1990 00:00:00 GMT","pragma":"no-cache","content-type":"application/json; charset=utf-8","vary":"X-Origin, Referer, Origin,Accept-Encoding","server":"scaffolding on HTTPServer2","x-xss-protection":"0","x-frame-options":"SAMEORIGIN","x-content-type-options":"nosniff","alt-svc":"h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","accept-ranges":"none","transfer-encoding":"chunked"}
[debug] [2020-12-09T09:24:15.780Z] >>> [apiv2][query] PUT https://dbname.firebaseio.com/.settings/defaultWriteSizeLimit.json [none]
[debug] [2020-12-09T09:24:15.780Z] >>> [apiv2][body] PUT https://dbname.firebaseio.com/.settings/defaultWriteSizeLimit.json "unlimited"
[debug] [2020-12-09T09:24:16.568Z] <<< [apiv2][status] PUT https://dbname.firebaseio.com/.settings/defaultWriteSizeLimit.json 400
[debug] [2020-12-09T09:24:16.568Z] <<< [apiv2][body] PUT https://dbname.firebaseio.com/.settings/defaultWriteSizeLimit.json {"error":"defaultWriteSizeLimit should be one of {\"large\", \"medium\", \"small\", \"unlimited\"}"}
[debug] [2020-12-09T09:24:16.907Z] FirebaseError: HTTP Error: 400, defaultWriteSizeLimit should be one of {"large", "medium", "small", "unlimited"}
    at module.exports (/Users/emrekhan/.cache/firebase/tools/lib/node_modules/firebase-tools/lib/responseToError.js:38:12)
    at Client.<anonymous> (/Users/emrekhan/.cache/firebase/tools/lib/node_modules/firebase-tools/lib/apiv2.js:191:27)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/emrekhan/.cache/firebase/tools/lib/node_modules/firebase-tools/lib/apiv2.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
[error] 
[error] Error: Unexpected error fetching configs at defaultWriteSizeLimit
firebase-cli firebase-realtime-database firebase
1 Answers

I was able to fix this problem for myself by patching firebase-tools: in database/settings.js I replaced line 23 with return JSON.stringify(input) and updated the setting successfully.

This has also been reported to Firebase Tools Issues on Github and fixed and merged today in this Pull Request

I assume we can expect a version bump on firebase-tools and then running

firebase database:settings:set defaultWriteSizeLimit unlimited

should work after updating to the latest version.

user_2361350
Related