How to populate a list with text from website

user_16088657 0
Viewed 43

I'm using this to populate a list(wordlist) from Amazon website product page hoping to get this text "Haz clic para obtener una vista ampliada" :

wordlist.append(WebDriverWait(robotel, 30).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span#canvasCaption"))).text)

Problem is that when I do it in a loop, for multiple product pages sometimes for some products wordlist is empty, but then if I test again for the same product page wordlist gets populated.

I can't fiind the reason for this, so please help.

selenium-webdriver selenium python-3.x python
1 Answers

Your line of code seems just perfect.

However, to avoid the empty wordlist you can additionally implement either/all of the following approaches:

  • As you are iterating through a loop some product pages and some products may take longer time to render, so you may need to increase the waiting time for 30 secs to 45 secs,
  • Beside considering the id attribute you can append some other attributes as well, e.g. class, aria-label, href, data-a-hires, alt, etc
user_74294470
Related