How webdriver communicates/works with browser?

user_6928227 0
Viewed 2029

enter image description here

I'm using Selenium for web crawler, it works fine at most time, but some websites can detect it, so I decided to learn more deeply.

After some search, i found "Chrome DevTools Protocol" and "Json Wire Protocol".

"Json Wire Protocol" works between Selenium and Webdriver as implementations for Python Java C# and other languages, so they can communicate with Webdriver by unified protocol, several articles explained this point.

But I can't find any article about how Webdriver communicate with browser, a few article says ChromeDriver communicates with Chrome by "Chrome DevTools Protocol", but they didn't explain details, so I'm not sure is this point correct.

How Browser receives command from Browser Driver and executes it?

selenium-chromedriver selenium-webdriver webdriver selenium
3 Answers

WebDriver basically lets you define driver object for 7 known browsers.

Webdriver is an interface and RemoteWebDriver is class that implements Webdriver Interface.

All mentioned 7 classes (chromedriver, safaridriver, edgeDriver and so) extend RemoteWebDriver class.

Below is the communication flow between webdriver and broswer :

  • for each Selenium command, a HTTP request is created and sent to the browser driver

  • the browser driver uses a HTTP server for getting the HTTP requests

  • the HTTP server determines the steps needed for implementing the Selenium command

  • the implementation steps are executed on the browser

  • the execution status is sent back to the HTTP server

  • the HTTP server sends the status back to the automation script

Read more about it here

enter image description here

user_94440730

On a high level selenium webdriver interacts with browser and it will not translate to Javascripts command, Basically our Java or Python Code will be sent as an api get and post request in JSON wire protocol. as explained in the above answer browser webdriver interacts with the real browser as a HTTP Request.

Every Browser Driver uses an HTTP server to receive HTTP requests. Once the URL reaches the Browser Driver, then it will pass that request to the real browser over HTTP. Once done, the commands in your Selenium script will be executed on the browser.

If the request is POST request, then there will be an action on the browser. If the request is a GET request then the corresponding response will be generated at the browser end. It will be then sent over HTTP to the browser driver and the Browser Driver over JSON Wire Protocol and sends it to the UI.

  • Test commands are converted into an HTTP request by the JSON wire protocol
  • Before executing any test cases, every browser has its own driver which initializes the server.
  • The browser then starts receiving the request through its driver.

for more information please refer the below site

BrowserStack :- https://www.browserstack.com/guide/selenium-webdriver-tutorial

Selenium Dev:- https://www.selenium.dev/documentation/en/webdriver/understanding_the_components/

Edureka:- https://www.edureka.co/blog/selenium-webdriver-architecture/

user_98923890

Two popular facilities used by web automated testing framework are WebDriver and DevTools. Before going in detail let us take a look some of the entities we will be talking.

  • WebDriver
  • WebDriver protocol
  • DevTools
  • DevTools Protocol
  • Chrome DevTools Protocol

WebDriver is a browser specific module (kind of a remote agent) that we (or testing libraries such as Selenium etc) can interact with it by using WebDriver protocol (it is a language-neutral wire protocol). The instructions we given to WebDriver will be delivered to it's browser by using its own proprietary communication mechanism. The WebDriver protocol consist of set of commands that abstract away common interactions with an application such as navigating, clicking, or reading the state of an element etc. Since it is a web standard, it is well supported across all major browser vendors. The WebDriver protocol is organized into commands; each HTTP request with a method and template defined in this specification represents a single command, and therefore each command produces a single HTTP response.

The DevTools is proprietary browser native interface that is being used to debug the browser from a remote application. The Chrome DevTools is from Chromium and the protocol used in this case is Chrome DevTools Protocol. Nowadays more and more browsers are adopting the Chrome DevTools Protocol to interact with their browser DevTools. The DevTools approach offers more automation capabilities to the automation tools than using WebDriver. We (or testing framework) could use this facility to interact directly (without any proxy or agent) with the web browser and Chrome DevTools is using WebSockets for this channel of communication. Then it is a bidirectional persistent connection between our testing framework module and the web browser.

Let us take a look on output snippet produced by the following Selenium node.js source code. We could see that it is using ws://127.0.0.1:51315, a WebSockets port 51315 on local host (by DevTools).

DevTools listening on 
ws://127.0.0.1:51315/devtools/browser/1c6fd882-e20d-44e9-9a86-bebcf93a514e

var webdriver = require('selenium-webdriver');

var driver = new webdriver.Builder().
      withCapabilities(webdriver.Capabilities.chrome()).
      build();
driver.get('https://www.amazon.com/');
driver.quit();
user_21578640
Related