Access to the Google Cloud Storage Trigger Events "Pub/Sub"?

user_6030951 0
Viewed 1848

I have a Google Cloud Storage Trigger set up on a Cloud Function with max instances of 5, to fire on the google.storage.object.finalize event of a Cloud Storage Bucket. The docs state that these events are "based on" the Cloud Pub/Sub.

Does anyone know:

  • Is there any way to see configuration of the topic or subscription in the console, or through the CLI?
  • Is there any way to get the queue depth (or equivalent?)
  • Is there any way to clear events?
google-cloud-pubsub google-cloud-functions google-cloud-storage google-cloud-platform
2 Answers

No, No and No. When you plug Cloud Functions to Cloud Storage event, all the stuff are handle behind the scene by Google and you see nothing and you can't interact with anything.

However, you can change the notification mechanism. Instead of plugin directly your Cloud Functions on Cloud Storage Event, plug a PubSub on your Cloud Storage event.

From there, you have access to YOUR pubsub. Monitor the queue, purge it, create the subscription that you want,...

user_113725930

The recomended way to work with storage notifications is using Pubsub.

Legacy storage notifications still work, but with pubsub you can "peek" into the pubsub message queue and clear it if you need it.

Also, you can process pubsub events with cloud run - which is easier to develop and test (just web service), easier to deploy (just a container) and it can process several requests in parallel without having to pay more (great when you have a lot of requests together).

Where does pubsub storage notifications go?

You can see where gcloud notifications go with the gsutil command:

% gsutil notification list gs://__bucket_name__
projects/_/buckets/__bucket_name__/notificationConfigs/1
        Cloud Pub/Sub topic: projects/__project_name__/topics/__topic_name__
        Filters:
                Event Types: OBJECT_FINALIZE

Is there any way to get the queue depth (or equivalent?)

In pubsub you can have many subsciptions to topics. If there is no subsciption, messages get lost.

To send data to a cloud function or cloud run you setup a push subscription. In my experience, you won't be able to see what happened because it faster that you can click: you'll find this empty 99.9999% of the time.

You can check the "queue" depht in the console (pubsub -> choose you topics -> choose the subscription).

If you need to troubleshoot this, set up a second subscription with a time to live low enough that it does not use a lot of space (you'll be billed for it).

Is there any way to clear events?

You can empty the messages from the pubsub subscription, but... ... if you're using a push notification agains a cloud function it will much faster than you can "click".

If you need it, it is on the web console (opent the pubsub subscription and click in the vertical "..." on the top right).

user_67603720
Related