Scraping etoro with python

user_13626448 0
Viewed 1046

I'm trying to use Selenium to automatically connect to my etoro account and get some figures from my portfolio. I work on Google Colab and from now, here is what I have:

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

wd = webdriver.Chrome(options=options)
wd.get("https://www.etoro.com/fr/login")

username = "username@mail.com"
password = "password"

elementID = wd.find_element_by_css_selector("input[automation-id='login-sts-username-input']")
elementID.send_keys(username)
elementID = wd.find_element_by_id("password")
elementID.send_keys(password)

However, I have this error message

Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[automation-id='login-sts-username-input']"}
  (Session info: headless chrome=91.0.4472.77)

I have tried to change and use find_element_by_class, by_xpath, etc but I couldn't find how to do it.

Could you give me a hand here?

web-scraping finance selenium css python
3 Answers

If you do not need to run with headless mode, removing that argument fixes this. In any case, that is where your problem lies.

I suspect that in headless browser mode we may be encountering some kind of bot detection. If you NEED to run in headless perhaps we can find a way around that.

user_81630710

See if this works:-

driver.find_element_by_xpath(".//input[@id='username']")
user_158251250
user_176428940
Related