I'm trying to use webscraping (via Python and Selenium) to create a worksheet with companies of interest to my boss. Most of it is working, I just can't seem to get hold of the "Next Page" button. Relative and absolute XPaths, CSS selectors, nothing seems to work, since every time you generate/switch pages they're diferent. (The relative XPath usually is '//*[@id="ember{SOME RANDOM NUMBER}"]') What could I do? There are other buttons with the same relative XPath structure in the page. 
Python Selenium LinkedIn company webscraping
Viewed 67
2 Answers
The Next page button has the same XPath for all the pages.
It is //button[@aria-label="Next"]
You should locate this element according to the aria-label attribute, not the id attribute value.
You can use class_name function to locate 'Next' element
next_button = wd.find_element_by_class_name('artdeco-pagination__button next').click()
Related