Error timeout in Linux server when a site is being crawling having "google invisible recaptcha"

Viewed 93

For the past few years I have been crawling the site. Suddenly "google invisible re-captcha" was implemented in a site.

In local machine (window) I got html content but when I run in Linux server "the timeout error" was occur. I've tried 5-10 min wait time but still not working.

Linux Debain (Cenots 7)

  • Chrome Version : 92.0.4515.107

         //String chromeDriverPath = "D:/CodeAxis/ChromeDriver/chromedriver.exe";// local machine driver path
         String chromeDriverPath = "/root/TestCaseChromeDriver/DriverPath/chromedriver";// linux driver server  path
    
         System.setProperty("webdriver.chrome.driver", chromeDriverPath);
    
         ChromeOptions options = new ChromeOptions();
         options.addArguments("--headless","--window-size=1920,1200","--ignore-certificate-errors");
         options.addArguments("start-maximized"); 
         options.addArguments("enable-automation");
         options.addArguments("--no-sandbox"); 
         options.addArguments("--disable-infobars");
         options.addArguments("--disable-dev-shm-usage"); 
         options.addArguments("--disable-browser-side-navigation"); 
         options.addArguments("--disable-gpu"); 
         options.addArguments("--disable-features=VizDisplayCompositor");
         options.addArguments("--disable-web-security");
         options.addArguments("user-agent=Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Mobile Safari/537.36 Edge/12.10166")
    
         WebDriver driver = new ChromeDriver(options);
    
         driver.manage().timeouts().implicitlyWait(120, TimeUnit.SECONDS);//if takes this much time to load all components
    
         driver.manage().timeouts().pageLoadTimeout(140, TimeUnit.SECONDS);  //Time allowed to load the page if not loaded
    
         driver.get("<SITE_PATH>");
         driver.getCurrentUrl();
         System.out.println(driver.getPageSource());
    
         driver.quit();
    
0 Answers
Related