How to use PubSub Rest API with Google service accounts?

user_3414253 0
Viewed 3615

Is there anyway by which we can call the Google PubSub Rest API using the Google service account ?

I have a very specific requirement to publish messages on pub sub using the rest api, let me know is there any way with which it can be done ?

If there is a way to get the required Api Key or token for rest api using the service account that helps as well.

google-cloud-pubsub rest
3 Answers

If you want to use the API directly, without the client library, you can do like this

Firstly, get the access_token and then add it in the header of your request

    const {GoogleAuth} = require('google-auth-library');
    const auth = new GoogleAuth()
    const adc = await auth.getApplicationDefault()
    const access_token = await adc.credential.getAccessToken();
    console.log(access_token.token)

    const request = require('request');

    const topic = 'projects/gbl-imt-homerider-basguillaueb/topics/go-hello'

    const options = {
        url: ' https://pubsub.googleapis.com/v1/' + topic + ':publish',
        method: 'POST',
        headers: {
            'Authorization': 'Bearer ' + access_token.token,
            'Content-type': 'application/json'
        },
        body:'{"messages":[{"data":"' + Buffer.from("Hello World").toString('base64') + '"}]}'
    };

    request(options, function(err, res, body) {
        let json = JSON.parse(body);
        console.log(json);
    });

EDIT

I'm sadly too bad in Nodejs to achieve a clean code for this. You can go to the documentation page and you can see there is constructor options for GoogleAuth class

At the end something like this should work (but I didn't achieve this... too bad...)

const auth = new GoogleAuth({
    scopes: '',
    keyFile: 'path/to/key.json'
})

Because I'm bad, I'm doing this and it works

//Set manually the environment variables
process.env.GOOGLE_APPLICATION_CREDENTIALS='path/to/key.json'
const auth = new GoogleAuth() //Use default credentials
user_113725930

Here is an example restApi call from AWS Lambda function to GCP pub/sub. I tried to find a way to pass the service_account.json file to GOOGLE_APPLICATION_CREDENTIALS environment variable on my lambda function but unfortunately couldn't find it so I decided to go rest API version instead of the SDK version. So I extract the client email and the private key from service_account.json(I added it on the AWS secrets manager and fetch it from there)

  const { JWT } = require("google-auth-library");
  const topicName = "test-topic";

  const client = new JWT({
    email: process.env.GCP_CLIENT_EMAIL,
    key: process.env.GCP_PRIVATE_KEY,
    scopes: ["https://www.googleapis.com/auth/cloud-platform"],
  });

  module.exports.publisher = async (event) => {
    console.log(event.body);
    const dataBuffer = Buffer.from(event.body).toString('base64');
    const url = `https://pubsub.googleapis.com/v1/${topicName}:publish`;
    const options = {
      url: url,
      method: 'POST',
      data: {
        messages: [
          {
            data: dataBuffer
          }
        ]
      }
    }
    const res = await client.request(options);
    console.log(res.data);
    return res.data;
  };
user_79019580

In order to use pub sub, You need to create IAM service account and enable pubsub service in google cloud console. Use the following function to publish and subscribe.

Publish :

const pubsub = new PubSub()  
const data = 'this is data : ' + new Date().toString();
     const dataBuffer = Buffer.from(data);
     const topicName = 'ypur-topic-name';

     pubsub
       .topic(topicName)
       .publisher()
       .publish(dataBuffer)
       .then((messageId) => {
         console.log(`Message ${messageId} published.`);
         return res.status(200).json({ success: true });
       })
       .catch((err) => {
         console.error('ERROR:', err);
       });

Subscribe :

const pubsub = new PubSub();

    const subscriptionName = 'your-subscription-name';
    const timeout = 6000;

    const subscription = pubsub.subscription(subscriptionName);
    let messageCount = 0;

    const messageHandler = (message) => {
      console.log(`Received message ${message.id}:`);
      console.log(`Data: ${message.data}`);
      console.log(`tAttributes: ${JSON.stringify(message.attributes)}`);
      messageCount += 1;

      // Ack the messae
      message.ack();
    };

    // Listen for new messages until timeout is hit
    subscription.on(`message`, messageHandler);
    setTimeout(() => {
      subscription.removeListener('message', messageHandler);
      console.log(`${messageCount} message(s) received.`);
    }, timeout * 1000);

You can refer to this tutorial for complete flow : https://blog.cloudboost.io/google-pubsub-tutorial-74dd5b948700

user_105675400
Related