How to Scrape This Field?

user_14358886 0
Viewed 43

My code goes to the webpage and scrapes all of the authors, but I need the session name as well.

My current code will print just "SESSION" and not the value for the session

from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
wait=WebDriverWait(driver, 70)

driver.get('https://meetinglibrary.asco.org/results?filters=JTVCJTdCJTIyZmllbGQlMjIlM0ElMjJmY3RNZWV0aW5nTmFtZSUyMiUyQyUyMnZhbHVlJTIyJTNBJTIyQVNDTyUyMEFubnVhbCUyME1lZXRpbmclMjIlMkMlMjJxdWVyeVZhbHVlJTIyJTNBJTIyQVNDTyUyMEFubnVhbCUyME1lZXRpbmclMjIlMkMlMjJjaGlsZHJlbiUyMiUzQSU1QiU1RCUyQyUyMmluZGV4JTIyJTNBMCUyQyUyMm5lc3RlZFBhdGglMjIlM0ElMjIwJTIyJTdEJTJDJTdCJTIyZmllbGQlMjIlM0ElMjJZZWFyJTIyJTJDJTIydmFsdWUlMjIlM0ElMjIyMDIxJTIyJTJDJTIycXVlcnlWYWx1ZSUyMiUzQSUyMjIwMjElMjIlMkMlMjJjaGlsZHJlbiUyMiUzQSU1QiU1RCUyQyUyMmluZGV4JTIyJTNBMSUyQyUyMm5lc3RlZFBhdGglMjIlM0ElMjIxJTIyJTdEJTVE')


productList=wait.until(EC.presence_of_all_elements_located((By.XPATH,"//div[@class='record']")))
for product in productList:
        author=product.find_element_by_xpath(".//div[@class='record__ellipsis']").text
        session=product.find_element_by_xpath("//div[@class='record__meta__title']").text
        print(session)
web-scraping selenium xpath python
2 Answers

To get the session name use xpath:

//div[@class='record__meta ng-star-inserted'][1]/div[2]

To get session header, I'd use:

//div[@class='record__category ng-star-inserted']

And title:

//div[@class='record__title ng-star-inserted']/span
user_127301120

For session try with this xpath .//div[@class='record__meta ng-star-inserted' and contains(., 'Session')]:

session=product.find_element_by_xpath(".//div[@class='record__meta ng-star-inserted' and contains(., 'Session')]").text
user_87520620
Related