How can I validate that a website sends a 2XX response using selenium in perl?

user_7019069 0
Viewed 184

as far as I understand, there is no way in selenium to get the response code of the website. How can I work it around to know if a website has sent me an error or exception without having to expect an element in the site and wait for it until it times out?

use strict;
use warnings;
use Selenium::Chrome;
use Selenium::Waiter qw/wait_until/;

my $chrome_driver_path = "./../../tools/drivers/chromedriver.exe";
my $driver;
my %settings = (
    'binary' => $chrome_driver_path,
);

$driver = Selenium::Chrome->new(%settings);

print("Getting stackoverflow\n");
wait_until{$driver->get("https://www.stackoverflow.com")};
validate_site($driver);

print("Getting unexistent url of stackoverflow\n");
wait_until{$driver->get("https://www.stackoverflow.com/this-does-not-exists-and-returns-404")};
validate_site($driver);


sleep(20);
$driver->shutdown_binary;

sub validate_site{
    my ($driver) = @_;
    #if ($driver->something) {
        # print("Looks good)\n");
    #}else{
        # warn("Error\n");
    # }
}

Expected result:

Getting stackoverflow:

Looks good

Getting unexistent url of stackoverflow:

Error

PD: I want to use selenium because I´m working in websites with javascript and storing cookies through different views, this is just an example to illustrate the problem that could be solved with a curl, but is not the case in my project.

selenium-webdriver selenium perl
2 Answers

Based on solution from your previous comment here's another solution. Here extra_capabilities are used to enable more logging (please note I added additional package). This will work in version 1.38 of Selenium::Remote::Driver that was released just recently, so you will need to update you packages if you haven't done so yet. This solution does not require falling back to WD2.

use strict;
use warnings;
use Selenium::Chrome;
use Selenium::Waiter qw/wait_until/;
use JSON;


my $chrome_driver_path = "./../../tools/drivers/chromedriver.exe";
my $driver;
my %settings = (
    'binary' => $chrome_driver_path,
    
    'extra_capabilities' =>{
      'goog:loggingPrefs' => {
          'performance' => 'ALL',
      },
      'goog:chromeOptions' => {
          'perfLoggingPrefs' => {
              'traceCategories' => 'performance',
          },
      },
    }
);


$driver = Selenium::Chrome->new(%settings);

print("Getting stackoverflow\n");
wait_until{$driver->get("https://www.stackoverflow.com")};
validate_site($driver);

print("Getting unexistent url of stackoverflow\n");
wait_until{$driver->get("https://www.stackoverflow.com/this-does-not-exists-and-returns-404")};
validate_site($driver);

#sleep(20);
$driver->shutdown_binary;

sub validate_site{
    my ($driver) = @_;
    
    my $logs = $driver->get_log('performance');
    my @responses = grep {$_->{'message'} =~ /"Network\.responseReceived"/ } @$logs;
    my @stat = grep {$_->{'message'} =~ $driver->get_current_url() } @responses;
    my $json= decode_json $stat[0]->{'message'};
    my $status = $json->{'message'}->{'params'}->{'response'}->{'status'};
    
    if ($status==200) {
         print("Looks good)\n");
    }else{
         warn("Error\n");
    }

}

In version 1.37 you would have to fallback to WD2 as goog:loggingPrefs capability was not supoprted:

use strict;
use warnings;
use Selenium::Chrome;
use Selenium::Waiter qw/wait_until/;
use JSON;


my $chrome_driver_path = "./../../tools/drivers/chromedriver.exe";
my $driver;
my %settings = (
    'binary' => $chrome_driver_path,
    
    'extra_capabilities' =>{
        'loggingPrefs' => {
            #'browser' => 'ALL',
            #'driver' => 'ALL',
            'performance' => 'ALL'
        },
        'perfLoggingPrefs' => {
            'traceCategories' => 'performance'
        },    
    }
);

$Selenium::Remote::Driver::FORCE_WD2=1;
$driver = Selenium::Chrome->new(%settings);

print("Getting stackoverflow\n");
wait_until{$driver->get("https://www.stackoverflow.com")};
validate_site($driver);

print("Getting unexistent url of stackoverflow\n");
wait_until{$driver->get("https://www.stackoverflow.com/this-does-not-exists-and-returns-404")};
validate_site($driver);

#sleep(20);
$driver->shutdown_binary;

sub validate_site{
    my ($driver) = @_;
    
    my $logs = $driver->get_log('performance');
    my @responses = grep {$_->{'message'} =~ /"Network\.responseReceived"/ } @$logs;
    my @stat = grep {$_->{'message'} =~ $driver->get_current_url() } @responses;
    my $json= decode_json $stat[0]->{'message'};
    my $status = $json->{'message'}->{'params'}->{'response'}->{'status'};
    
    if ($status==200) {
         print("Looks good)\n");
    }else{
         warn("Error\n");
    }

}
user_106077320

As you mentioned, WebDriver does not expose http response codes and it is suggested to use proxy if you really need it .

If you do not want to wait for the element too long, you could reduce implicit timeouts in validate_site and look for reliable element eg:

sub validate_site{
    my ($driver) = @_;
    
    my $implicit=$driver->get_timeouts()->{implicit};# get current value
    $driver->set_implicit_wait_timeout(0);# set it to 0
    my @elem = $driver->find_elements(".py128","css");#this 'reliable element' it's present on https://www.stackoverflow.com but not on 404 page

    if (@elem) {
         print("Looks good)\n");
    }else{
         warn("Error\n");
    }

    $driver->set_implicit_wait_timeout($implicit);# restore original value
}

or if you really want to workaround it and don't mind duplicating requests you could try https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

sub validate_site{
    my ($driver) = @_;
    
    my $script = q{
       let page_url=window.document.URL;
       let resp = await fetch(page_url);
       return resp.status;
    }; 
    my $status = $driver->execute_script($script);

    if ($status==200) {
         print("Looks good)\n");
    }else{
         warn("Error\n");
    }
   
}

if you don't need the body (saves you response size) then you can request only headers by adding HEAD method (instead of default GET)

let resp = await fetch(page_url,{method:"HEAD"});

user_106077320
Related