Can you convert UrlFetchApp into a promise in Apps Script?

user_14485586 0
Viewed 693

UrlFetchApp.fetch executes synchronously and makes execution slow. Is it possible to convert UrlFetchApp into promise?

I have been thinking of this approach:

  1. Return HTTPResponse.getContent() as a Promise and add all the urls into a queue.
  2. Defer their execution till getContent().then() is invoked.
  3. When getContent() of any of the urls is called, use fetchAll to get all results and clear the queue.

Do you see any problems with this approach?

urlfetch promise google-apps-script
1 Answers

Theoretically, The approach seems sound, especially since it's known that .fetchAll executes asynchronously.

  • .fetch() call is where the actual fetch happens. So any hook into the UrlFetchApp object should be inserted before the fetch call is made.

  • You may employ a Proxy object to hook into .fetch calls on UrlFetchApp to return a dummy HTTPResponse with a thenable object

  • Then use fetchAll on .getContent call as stated in the question.

  • Note however that promises in apps script may or may not execute asynchronously as stated in the issue comments #1 to #4. This however shouldn't be a issue with your approach.

Given the finicky nature and unclear documentation on promises, It's better to avoid them in any production environment. A better way to achieve batching requests is to use plain custom functions with a thenable object:

function test() {
  /**
   * @description Batches requests until then is called on a response
   *   and fetches all batched requests
   * @return {object} A then object, which when called fetches the batch
   * @param {string} url Url to fetch
   * @param {object} options Options to fetch. See UrlFetchApp.fetch
   */
  const fetchAsBatch = function fetch(requests, url, options) {
    options.url = url;
    requests.add(options);
    return {
      then: func => {
        const responses = func(UrlFetchApp.fetchAll([...requests]));
        requests.clear();// clear current batch
        return responses;
      },
    };
  }.bind(this, new Set());

  const successHandlerLogger = responses => {
    /*Do something with all responses*/
    console.log(responses.map(response => response.getContentText()));
  };
  fetchAsBatch('https://example.com', { method: 'get' });
  fetchAsBatch('https://httpbin.org/post', { method: 'post' }).then(
    successHandlerLogger
  );
  fetchAsBatch('https://google.com', {}).then(successHandlerLogger);
}


function test() {
  /**
   * @description Batches requests until then is called on a response
   *   and fetches all batched requests
   * @return {object} A then object, which when called fetches the batch
   * @param {string} url Url to fetch
   * @param {object} options Options to fetch. See UrlFetchApp.fetch
   */
  const fetchAsBatch = function fetch(requests, url, options) {
    options.url = url;
    requests.add(options);
    return {
      then: func => {
        const responses = func(UrlFetchApp.fetchAll([...requests]));
        requests.clear();
        return responses;
      },
    };
  }.bind(this, new Set());

  const successHandlerLogger = responses => {
    /*Do something with all responses*/
    console.log(responses.map(response => response.getContentText()));
  };
  fetchAsBatch('https://example.com', { method: 'get' });
  fetchAsBatch('https://httpbin.org/post', { method: 'post' }).then(
    successHandlerLogger
  );
  fetchAsBatch('https://google.com', {}).then(successHandlerLogger);
}
/*Mock urlfetchapp library to return requests without fetch*/
const UrlFetchApp = {
  fetchAll: requests =>
    requests.map(request => ({
      getContentText: () => request,
    })),
};

test();

user_84044530
Related