I am scraping LinkedIn job search using Selenium. LinkedIn search result page is divided into two sections, left and right as follows: enter image description here
On the left, you see job titles, and on the right job descriptions. I have the following code to extract job listings from the left side:
wd.execute_script(“window.scrollTo(0, document.body.scrollHeight);”)
job_lists = wd.find_element_by_class_name(‘jobs-search__results-list’)
jobs = jobs_lists.find_elements_by_tag_name(‘li’) # return a list
However, once page loads, it sounds like the first line is not working and all job title are not loaded. Any idea on how to fix this is much appreciated.