How to use headless chrome in python on vercel?

Shahriar Zarir 0
Viewed 26

You may know that heroku will stop their free dyno, free postgres etc from November. So I was finding some alternative to run my python web apps. I have almost 10 regular web apps which I visit regularly, like: url shortener, keyword research, google drive direct link generator site and many more. All of these are hosted on heroku. But I'm moving to vercel now. I setup all projects on vercel but the last one is complicated. My last project is python selenium bot. This one is my keyword research web app. I used some buildpack eg: Headless Chrome (https://github.com/heroku/heroku-buildpack-google-chrome) and Chromedriver (https://github.com/heroku/heroku-buildpack-chromedriver) to make this project run properly. But the problem is I could not find anything like buildpack in vercel to add Chrome and Chromedriver.
Anyone know about this?

Edit:

That was a kind of story and many people didn’t understand what I was asking.

So, My project is about selenium (python). Selenium needs google chrome browser installed and a chromedriver to run itself. There is another option without installing chrome is to set chrome binary location in webdriver.ChromeOptions(). I want to host this selenium project on vercel.com which is linux based.

So my question is how can I install Chrome Browser and ChromeDriver in vercel?

heroku vercel selenium python
1 Answers
  • Your question is not very clear but as far as I have understand this is what you can do and all you need for webscraping and keyword density analyser can be downloaded from pip itself.

from selenium import webdriver
from bs4 import BeautifulSoup
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--headless")

browser = webdriver.Chrome(ChromeDriverManager().install(),chrome_options=chrome_options)

Additional info: For webscraper if you want to scrape via URL then you can use something like this in flask-restful:

class WebpageScraper(Resource):
    def get(self):
        URL=request.args['url']
        browser.get(URL)
        html_source = browser.page_source  
        page = BeautifulSoup(html_source, "html.parser")
        for data in page(['style', 'script']):
            data.decompose()
        data_out=' '.join(page.stripped_strings).replace("\xa0", " ")
        return Response(data_out,mimetype="text")
Harshan GAK0
Related