I've written a tiny script in python using xpath to scrape the titles, being displayed in the left sided area, from yahoo finance. There are several titles in that page but when I run my script I only get three titles. I do not wish to use selenium in this case as I'm getting at least the minimum result. I tried to use "time" in my script to let the webpage load in full. However, it seems that "time" has got nothing to do with it. How can I get all the titles from that webpage other than using selenium? Here is what I tried:
import time
import requests
from lxml import html
response = requests.get("https://finance.yahoo.com/").text
time.sleep(5)
root = html.fromstring(response)
for title in root.xpath("//a[u[@class='StretchedBox']]/text()"):
print(title)