GCP Pub/Sub & Python - How To Acquire JSON Keys From Message?

user_10005028 0
Viewed 1330

I must apologise first as I'm just starting out with Pub/Sub so my understanding of the GCP feature is likely lacking.

I'm trying to adapt Google's own Python script of receiving a message from Pub/Sub within Cloud Run (https://cloud.google.com/run/docs/triggering/pubsub-push#run_pubsub_handler-python):

@app.route("/", methods=["POST"])
def index():
    envelope = request.get_json()
    if not envelope:
        msg = "no Pub/Sub message received"
        print(f"error: {msg}")
        return f"Bad Request: {msg}", 400

    if not isinstance(envelope, dict) or "message" not in envelope:
        msg = "invalid Pub/Sub message format"
        print(f"error: {msg}")
        return f"Bad Request: {msg}", 400

    pubsub_message = envelope["message"]

    name = "World"
    if isinstance(pubsub_message, dict) and "data" in pubsub_message:
        name = base64.b64decode(pubsub_message["data"]).decode("utf-8").strip()

    print(f"Hello {name}!")

    return ("", 204)

Now the Pub/Sub Topic I'm using has an AVRO schema like:

{
  "type": "record",
  "name": "Avro",
  "fields": [
    {
      "name": "ext_file_id",
      "type": "string"
    },
    {
      "name": "input_bucket",
      "type": "string"
    }
  ]
}

What I need to do within Cloud Run (my Python script) is to acquire the JSON body keys (ext_file_id and input_bucket), along with their values.

Considering I get the JSON via:

envelope = request.get_json()

Would it simply be something like:

envlope.body.ext_file_id
envelope.body.input_bucket

Or something else?

I've tried the following to no avail:

envelope["ext_file_id"]
envelope["input_bucket"]
google-cloud-pubsub google-cloud-platform python json
1 Answers

When you receive the JSON from PubSub, you receive a JSON with message field which contains this format

{
  "data": string,
  "attributes": {
    string: string,
    ...
  },
  "messageId": string,
  "publishTime": string,
  "orderingKey": string
}

Your message is encoded in base64 in the data field, in your code

pubsub_message["data"]).decode("utf-8")

Then you have to JSON parse (json.loads(data)) this content, and then get the field that you want.

user_113725930
Related