How to find element in nested classes in Selenium (python)?

nikhil bhathi 0
Viewed 32

I am trying to scrape reviews from this website: https://www.goodreads.com/book/show/4865.How_to_Win_Friends_and_Influence_People?from_search=true&from_srp=true&qid=zsfs3jEPvd&rank=1

Reviews are hidden down many nested classes, I am trying to reach them but facing issues. I am fairly new to selenium. So far, I tried:

'''
        a = driver.find_element("class name", "BookPage__reviewsSection")
        for i in a.find_element("xpath", "//* [@id='ReviewsSection']").find_elements("class name",'lazyload-wrapper '):
         print(i.find_element("xpath","//div[@class='ReviewsList']").text)
'''

The print statement outputs:

Friends & Following
Create a free account to discover what your friends think of this book!

Friends & Following
Create a free account to discover what your friends think of this book!

According to the output it just finds 'BookPage__reviewsSection' class and then 'ReviewsList' class which explains the output. Why doesn't it find 'lazyload-wrapper' class and then 'ReviewsList' class inside it?

I appreciate the help.

selenium python
1 Answers

@nikhil bhati, you can try the following code. Basically I directly took the Xpath for all the review comments. Let me know if this helps or you wanted some other output. Sorry I have not tried your way of finding the element.

driver.get("https://www.goodreads.com/book/show/4865.How_to_Win_Friends_and_Influence_People?from_search=true&from_srp=true&qid=zsfs3jEPvd&rank=1")
allReviewTexts = driver.find_elements("xpath", "//div[@id='other_reviews']//div[@id='bookReviews']//span[contains(@id, 'reviewTextContainer')]//span[contains(@id, 'freeTextContainer')]")
print(len(allReviewTexts))
for i in allReviewTexts:
    print(i.text)
ketanvj0
Related