Unable to programatically login to a website

user_15799288 0
Viewed 50

So I am trying to login programatically (python) to https://www.datacamp.com/users/sign_in using my email & password.

I have tried 2 methods of login. One using requests library & another using selenium (code below). Both time facing [403] issue.

Could someone please help me login programatically to it ?

Thank you !

Using Requests library.

import requests; r = requests.get("https://www.datacamp.com/users/sign_in"); r (which gives <response [403]>)

Using Selenium webdriver.

 driver = webdriver.Chrome(executable_path=driver_path, options=option)
 driver.get("https://www.datacamp.com/users/sign_in")
 driver.find_element_by_id("user_email") # there is supposed to be form element with id=user_email for inputting email
python-requests web-scraping selenium python
2 Answers

Implicit wait at least should have worked, like this:

from selenium import webdriver


driver = webdriver.Chrome(executable_path='/snap/bin/chromium.chromedriver')
driver.implicitly_wait(10)
url = "https://www.datacamp.com/users/sign_in"
driver.get(url)
driver.find_element_by_id("user_email").send_keys("test@dsfdfs.com")
driver.find_element_by_css_selector("#new_user>button[type=button]").click()

BUT

The real issue is the the site uses anti-scraping software. If you open Console and go to request itself you'll see: enter image description here

It means that the site blocks your connection even before you try to login. Here is similar question with different solutions: Can a website detect when you are using Selenium with chromedriver?

Not all answers will work for you, try different approaches suggested.

With Firefox you'll have the same issue (I've already checked).

user_127301120

You have to add a wait after driver.get("https://www.datacamp.com/users/sign_in") before driver.find_element_by_id("user_email") to let the page loaded.
Try something like WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'user_email')))

user_34854340
Related