Using WebDriver to take screenshots with always on process

Question

I have a service that takes screenshots of given url using Selenium Web Driver.

It workes Ok, raises a process -> takes the screenshot -> closes the process.

the problem is - it takes too long to return.

is there a way that the web driver process stays always-on and waits for requests?

here is my code

class WebDriver(webdriver.Chrome):
    def __init__(self, *args, **kwargs):
        logger.info('Start WebDriver instance.')
        self.start_time = datetime.now()
        self.lock = threading.Lock()
        kwargs['chrome_options'] = self.get_chrome_options()
        super().__init__(*args, **kwargs)

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        logger.info(f'Quiting Webdriver instance {id(self)}, took {datetime.now() - self.start_time}')
        self.quit()

    @staticmethod
    def get_chrome_options():
        chrome_options = ChromeOptions()
        chrome_options.headless = True
        chrome_options.add_argument('--start-maximized')
        chrome_options.add_argument("--no-sandbox")  # Bypass OS security model
        chrome_options.add_argument('--disable-dev-shm-usage')  # overcome limited resource problems
        chrome_options.add_argument("--lang=en")
        chrome_options.add_argument("--disable-infobars")  # disabling infobars
        chrome_options.add_argument("--disable-extensions")  # disabling extensions
        chrome_options.add_argument("--hide-scrollbars")
        return chrome_options

    def capture_screenshot_from_html_string(self, html_str, window_size):
        with tempfile.TemporaryDirectory() as tmpdirname:
            html_filename = tmpdirname + f'/template.html'
            with open(html_filename, 'w') as f:
                f.write(html_str)

            url = 'file://' + html_filename
            img_str = self.capture_screenshot(url, window_size)

        return img_str

    def capture_screenshot(self, url, window_size):
        self.lock.acquire()
        try:
            self.set_window_size(*window_size)
            self.get(url)
            self.maximize_window()
            self.set_page_load_timeout(PAGE_LOAD_TIMEOUT)

            img_str = self.get_screenshot_as_png()

        except Exception as exc:
            logger.error(f'Error capturing screenshot url: {url}; {exc}')
            img_str = None

        finally:
            self.lock.release()

        return img_str

user_15083356 · Answer

After some research i found a solution and im posting it to maybe help others in similar problem.

using py-object-pool library.

Object pool library creates a pool of resource class instance and use them in your project. Pool is implemented using python built in library Queue.

Each time creating a new browser instance is time consuming task which will make client to wait. If you have one browser instance and manage with browser tab, it will become cumbersome to maintain and debug in case of any issue arises. Object Pool will help you to manage in that situation as it creates resource pool and provides to each client when it requests. Thus separating the process from one another without waiting or creating new instance on the spot.

Code Example


ff_browser_pool = ObjectPool(FirefoxBrowser, min_init=2)

with ff_browser_pool.get() as (browser, browser_stats):
    title = browser.get_page_title('https://www.google.co.in/')

for more information see link below https://pypi.org/project/py-object-pool/

Using WebDriver to take screenshots with always on process

1 Answers

Code Example