How can I call arguments and return the outputs from a Google cloud hosted function?

user_13259964 0
Viewed 155

I want to write a python function that allows me to scrape a website given a url, that will interface with a flutter mobile application. I have written the very basic code on python, but now I have no clue whatsoever on how to integrate it Google Cloud. Would really appreciate some walkthrough help.

My code model is as follows:

import timeago
import pytz
from bs4 import BeautifulSoup
from time import mktime
from datetime import datetime

def Parser(url):
     ......# URL PARSING
     ......
     ......
     return domain_name, page_title, clean_txt, timestamp
url = '******************'
file = Parser(url)
print (file[0], file [1], file [2], file [3])

I have tried looking at many examples online, and most of them use flask. I do not understand what flask is used for to be honest, and how to use it for Google Cloud Functions (or any cloud functions for that matter). As far as I understand, I should put my requirements.txt file as follows:

timeago
pytz
mktime
bs4
datetime

If so, then how should I define my imports (i.e. the from bs4 import BeautifulSoup, etc) statements?

Also, how can I call these functions to my flutter app code? I have the url for the cloud function, but don't know what to do with it. I tried going through the official documentation, but most of it is just "Hello World" code that is barely of any use to understand (for me), due to the lack of any input or output arguments.

Thanks and cheers

python-requests google-cloud-functions flutter flask python
1 Answers

To address some of your queries:

I do not understand what flask is used for to be honest, and how to use it for Google Cloud Functions (or any cloud functions for that matter)

There are a few methods to trigger a Cloud Function, one of the more common ones is using HTTP Triggers.

For HTTP Triggers, Flask is needed to wrap around your actual function/business logic with a RESTful endpoint, so that your function can be invoked via an HTTP request (The latter action should be performed by your flutter code). Some of this is actually handled by Cloud Functions, and you mainly need to specify a wrapper method taking in a flask.Request object (essentially the payload of the above-mentioned HTTP request) as an argument to invoke your main business logic/function, wherein the flask.Request object will be the point where your arguments (e.g. url) are provided. You will then need to provide this wrapper method when deploying the cloud function (see below).

If so, then how should I define my imports (i.e. the from bs4 import BeautifulSoup, etc) statements?

The import statements work as usual as per your code snippet.

how can I call these functions to my flutter app code?

You will need to fire an HTTP request from your flutter app code to the URL of the cloud function.

To put things into context:

In your example above, you can wrap it around an entry point method with a flask.Request object like this:

import timeago
import pytz
from bs4 import BeautifulSoup
from time import mktime
from datetime import datetime
from flask import escape


def Parser(url):
     ......# URL PARSING
     ......
     ......
     return domain_name, page_title, clean_txt, timestamp


def invoke_parser(request):
    request_json = request.get_json(silent=True)
    file = Parser(escape(request_json['url']))
    print (file[0], file [1], file [2], file [3])

How your function has changed is that url has to be passed in the HTTP request body instead when invoking the function.

So, to deploy the function, use the command:

gcloud functions deploy [CLOUD FUNCTION NAME] \
--entry-point invoke_parser \
--runtime python37 \
--trigger-http

The --entry-point argument will be the wrapper method to be invoked (I have used invoke_parser as the wrapper method in my example).

To invoke it, fire an HTTP request (this is where the url argument to your Parser function can be provided) e.g.:

curl -X POST \
"https://YOUR_REGION-YOUR_PROJECT_ID.cloudfunctions.net/FUNCTION_NAME" \
-H "Content-Type:application/json" \
--data '{"url":"******************"}'

In your flutter code, you will need to fire an HTTP request equivalent to the above curl command

user_65407620
Related