Table showing on web page but no where to be found in HTML. Python, Selenium

Matthew Rozanoff 0
Viewed 52

This is a private website, so I will try my best to explain it.

Scraping with Selenium, and table is not in the HTML at all, although it is visible to the user and interactable.

The HTML code on the page is accessed through a navbar click, and then the HTML is pretty simple. Has three or so divs, and some scripts.

<head></head>
<body>
 <div></div>
 <div></div>
</body>

But the actual webpage has an entire dynamic table that opens reports in new windows when clicked on as well as a bunch of other stuff.

Why can I see everything on the webpage, but not in Selenium OR with Inspect Element? (unless I inspect search bar)

The missing HTML showed up when I clicked inspect element on the search bar, and from there I was able to view the HTML, but that isn't a solution for Selenium.

I dont know if this helps, but the ID's of the hidden HTML have this in it: 'crmGrid_visualizationCompositeControl'

Thank you!

selenium javascript html python
1 Answers

Makes sense, Selenium does NOT know how to find elements inside an Iframe. You must tell the Selenium to switch into the Iframe so it will be able to "see" the elements inside:

iframe = driver.find_element_by_xpath("//iframe[@name='Name_Of_Iframe']")
driver.switch_to.frame(iframe)

If the Iframe has an Id - find it by Id. You can also switch to other iframes by index:

iframe = driver.find_elements_by_tag_name('iframe')[2]
driver.switch_to.frame(iframe)
Tal Angel0
Related