Web-scraping using Selenium: Get current url after selecting dropdown menu

user_12863170 0
Viewed 41

I am trying to scrape pricing information for clothes from Amazon. But I have to select the clothes size. After selecting the size needed, how do I keep track of the new URL? The following code is working and selecting the first value in the dropdown menu. But I just don't know how to keep track of the new url.

original url: https://www.amazon.ae/Jack-Jones-Glenn-Original-Pants/dp/B07JQ8MDGD/ref=sr_1_5?crid=M8QQKGLLZ1O9&keywords=jeans&qid=1657289288&sprefix=jeans%2Caps%2C232&sr=8-5&th=1

url after selecting size (the url I want to get): https://www.amazon.ae/Jack-Jones-Glenn-Original-Pants/dp/B07JQBYC8J/ref=sr_1_5?crid=M8QQKGLLZ1O9&keywords=jeans&qid=1657289288&sprefix=jeans%2Caps%2C232&sr=8-5&th=1&psc=1

click here if you want to see the screenshot of the web-page

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import Select

url='https://www.amazon.ae/Jack-Jones-Glenn-Original-Pants/dp/B07JQB87KL/ref=sr_1_5? 
crid=M8QQKGLLZ1O9&keywords=jeans&qid=1657289288&sprefix=jeans%2Caps%2C232&sr=8- 
5&th=1'
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get(url)

select=Select(driver.find_element_by_id("native_dropdown_selected_size_name"))
select.select_by_index(2)
#driver.current_url: is returning the original url
selenium-webdriver python
1 Answers

Maybe selenium is moving on from the .select_by_index step to getting the URL before the site has a chance to change its own URL.

You might try Implicit Wait (based on time) :

driver.implicitly_wait(10)  # force driver to wait 10 seconds

Or Explicit Wait (based on expected condition):

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.ID, 'someid')))

Your expected condition will depend on your use case.

I would try the implicit wait first, just to see if you can get the updated driver.current_url

user_148462960
Related