How to make API calls idempotent in a Cloud Function trigger

Question

I have an onCreate cloud function that hits an API but I need to be sure that it never hits it twice. Cloud function triggers can execute more than once and we are encouraged to use the eventId for idempotency. Here is what I currently have:

    let ship_ref = admin.firestore().collection('shipments').doc(event_id)
    return admin.firestore().runTransaction(transaction => {
      return transaction.get(ship_ref)
      .then(ship_doc => {
        if (ship_doc.exists) {
          console.log('label already generated')
          return null 
        }
        // generate the label
        return axios({
          method: 'post',
          ...rest of the axios config
        })
        .then(label_response => {
          const label_data = label_response.data

          // add the label data to the shipment collection
          transaction.set(ship_ref, {
            ...sale_data,
            label_data,
            tracking_url_data,
            completed_sale_ref: snap.ref
          })
          return null
        })
        .catch(e => {
          // handle fail
        }) 
      })
    })
    .then(() => {
      // transaction complete
    })
    .catch(e => {
      // error
    })

Here is what I am concerned about. As per the docs a firestore transaction will run again if the contents of the document are affected by a concurrent action. In this case if it "runs again" it will run the axios call again and that's bad for me.

If a cloud function trigger (onCreate in this case) runs more than once, is there ever a chance that it will be concurrent and cause a race condition? If so, then it is possible that the axios request could be made more than one time and that's what I have to avoid.

Thank you in advance for your help.

user_807126 · Answer

If a cloud function trigger (onCreate in this case) runs more than once, is there ever a chance that it will be concurrent and cause a race condition?

You can assume that retries will not overlap prior invocations. Think of it this way: if you called someone on the phone, and they didn't respond, calling them again concurrently wouldn't really improve your chance of getting a hold of them. You'd wait some time after hanging up before trying again. That's kinda what's going on here. Cloud Functions might try to invoke the function again if a message between it and the function got dropped the first time for some reason.

Aside from all that, you shouldn't be performing stateful work inside a transaction. That's very bad on its own right. The transaction itself has its own retry mechanism in case there is contention on the documents in that transaction. Transaction handlers should be fully stateless and only operate on static data outside of the documents being transacted upon. This is stated in the documentation:

When using transactions, note that ... Transaction functions should not directly modify application state.

Also:

Do not modify application state inside of your transaction functions. Doing so will introduce concurrency issues, because transaction functions can run multiple times and are not guaranteed to run on the UI thread. Instead, pass information you need out of your transaction functions.

The call to axios should be considered application state, especially since it's a post. I would go further and say that transaction handlers also should not block at all. They need to run as fast as possible in order to reduce contention.

How to make API calls idempotent in a Cloud Function trigger

1 Answers