Google Cloud Platform - Deploy a Cloud Function that starts a webdriver

user_10079397 0
Viewed 4795

I am defining a Cloud Function on GCP for scraping a website in Python.

I am starting simple by defining a function that simply opens the webdriver:

from selenium import webdriver

def launch_search(request):
    # Starting a webdriver
    driver = webdriver.Chrome()
    return 'Success'

This function doesn't work (Error: could not handle the request when I trigger it), probably because the Chrome Driver is not installed on my remote machine. Therefore:

  • How can I install it?
  • Or can I scrape a webpage using Selenium, without opening the page with a webdriver?
web-scraping google-cloud-platform selenium automation python
3 Answers

Given that Cloud Functions are Serverless, you are unable to control the server machine. You can use other manageable services such as GCE or GKE

user_11191530

You can't currently use Python to run Selenium scripts. There's a Feature Request in the Public Issue Tracker currently open for this that can be found here.

As an alternative, you can use Node.JS with Puppeteer. I found this blog post that details a use-case.

user_91468200
user_87590220
Related