Avoid triggering Firebase functions by real-time database on special cases

Viewed 342

Sometimes we use the firebase functions triggered by real-time database (onCreate/onDelete/onUpdate ...) to do some logic (like counting, etc).

My question, would it be possible to avoid this trigger in some cases. Mainly, when I would like to allow a user to import a huge JSON to firebase?

Example: a function E triggered on the creation of a new child in /examples. Normally, users add examples one by one to /examples and function E runs to do some logic. However, I would like to allow a user (from the front-end) to import 2000 children to /examples and the logic which is done by function E is possible at import time without the need for E. Then, I do not need E to be triggered for such a case where a high number of functions could be executed. (Note: I am aware of the 1000 limit)

Update: based on the accepted answer, submitted my answer down.

3 Answers

As far as I know, there is no way to disable a Cloud Function programmatically without just deleting it. However this introduces an edge case where data is added to the database while the import is taking place.

A compromise would be to signal that the data you are uploading should be post-processed. Let's say you were uploading to /examples/{pushId}, instead of attaching the database trigger to /examples/{pushId}, attach it to /examples/{pushId}/needsProcessing (or something similar). Unfortunately this has the trade-off of not being able to make use of change objects for onUpdate() and onWrite().

const result = await firebase.database.ref('/examples').push({
  title: "Example 1A",
  desc: "This is an example",
  attachments: { /* ... */ },
  class: "-MTjzAKMcJzhhtxwUbFw",
  author: "johndoe1970",
  needsProcessing: true
});
async function handleExampleProcessing(snapshot, context) {
  // do post processing if needsProcessing is truthy
  if (!snapshot.exists() || !snapshot.val()) {
    console.log('No processing needed, exiting.');
    return;
  }

  const exampleRef = admin.database().ref(change.ref.parent); // /examples/{pushId}, as admin
  const data = await exampleRef.once('value');

  // do something with data, like mutate it

  // commit changes
  return exampleRef.update({
    ...data,
    needsProcessing: null /* delete needsProcessing value */
  });
}

const functionsExampleProcessingRef = functions.database.ref("examples/{pushId}/needsProcessing");

export const handleExampleNeedingProcessingOnCreate = functionsExampleProcessingRef.onCreate(handleExampleProcessing);

// this is only needed if you ever intend on writing `needsProcessing = /* some falsy value */`, I recommend just creating and deleting it, then you can use just the above trigger.
export const handleExampleNeedingProcessingOnUpdate = functionsExampleProcessingRef.onUpdate((change, context) => handleExampleProcessing(change.after, context));

An alternative to Sam's approach is to use feature flags to determine if a Cloud Function performs its main function. I often have this in my code:

exports.onUpload = functions.database
  .ref("/uploads/{uploadId}")
  .onWrite((event) => {
  return ifEnabled("transcribe").then(() => {
    console.log("transcription is enabled: calling Cloud Speech");
    ...
  })
});

The ifEnabled is a simple helper function that checks (also in Realtime Database) if the feature is enabled:

function ifEnabled(feature) {
  console.log("Checking if feature '"+feature+"' is enabled");
  return new Promise((resolve, reject) => {
    admin.database().ref("/config/features")
      .child(feature)
      .once('value')
      .then(snapshot => {
        if (snapshot.val()) {
          resolve(snapshot.val());
        }
        else {
          reject("No value or 'falsy' value found");
        }
      });
  });
}

Most of my usage of this is during talks at conferences, to enable the Cloud Functions at the right time (as a deploy takes a bit longer than we'd like for a demo). But the same approach should work to temporarily disable features during for example data import.

Okay, another solution would be

A: Add a new table in firebase like /triggers-queue where all CRUD that should fire a background function are added. In this table, we add a key for each table that should have triggers - in our example /examples table. Any key that represents a table should also have /created, /updated, and /deleted keys as follows.

/examples
.../example-id-1

/triggers-queue
.../examples
....../created
........./example-id
....../updated
........./example-id
............old-value
....../deleted
........./example-id
............old-value

Note that the old-value should be added from app (front-end, etc). We set triggers always onCreate on /triggers-queue/examples/created/{exampleID} (simulate onCreate)

/triggers-queue/examples/updated/{exampleID} (simulate onUpdate)

/triggers-queue/examples/deleted/{exampleID} (simulate onDelete)

The fired function can know all the necessary info to handle the logic as follows:

  • Operation type: from the path (either: created, updated, or deleted)
  • key of the object: from the path
  • current data: by reading the corresponding table (i.e., /examples/id)
  • old data: from the triggers table

Good Points:

  • You can import a huge data to /examples table without firing any function as we do not add to the /triggers-queue
  • you can fanout functions to pass the limit 1000/sec. That is by setting triggers on (as an example to fanout on-create) /triggers-queue/examples/created0/{exampleID} and /triggers-queue/examples/created1/{exampleID}

bad-points:

  • more difficult to implement
  • need to write more data to firebase (like old-data) from the app.

B- Another way (although not an answer for this) is to move the login in the background function to an HTTP function and call it on every crud ops.

Related