I am trying to scrape delivery dates for items on shopee's shanghai website and I realized that the element only appears when I hover over "Shipping Fee" on each of the item.
I managed to freeze the javascript in the debugger during inspection mode and found that the delivery date element's xpath is '//div[@class="sYwpBA"]'
So what I did so far is use Selenium's ActionChains to hover over the "Shipping Fee" element and then make my driver wait until it finds the delivery date element.
However, it's not working and giving me blank which means my driver still isn't able to find the delivery date element.
from selenium import webdriver
from selenium.common import exceptions
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
chromedriver = "path to chromedriver"
driver = webdriver.Chrome(chromedriver)
url = https://shopee.sg/-Bundle-of-2-SOMEBYMI-150ml-AHA-BHA-PHA-30days-Miracle-Toner-150ml-i.93906301.7143079648?ads_keyword=wkdaelpmissisiht&adsid=3545314&campaignid=1926059&position=0
driver.get(url)
time.sleep(2)
try:
shipping = driver.find_element_by_xpath('//div/div[@class="shopee-drawer "]')
hov = ActionChains(driver).move_to_element(shipping)
hov.perform()
delivery_date = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//div[@class="sYwpBA"]')))
except (NoSuchElementException, StaleElementReferenceException):
delivery_date = None
# ROW
row = {
}
if delivery_date is not None:
row['Delivery Date'] = delivery_date.text
else:
row['Delivery Date'] = ""
df = pd.DataFrame(rows)
df.to_csv('delivery_date.csv', index=False)
What can I do to find the delivery date element that's only locatable when I hover over the Shipping Fee element?

