Python Selenium accessing Barchart show-all page

user_9863205 0
Viewed 60

I have a URL where I want Selenium to click on the "Show all" button at the bottom of the page. https://www.barchart.com/stocks/indices/sp/sp500?page=1

But I can't locate it in the HTML loaded. When I inspect the button, it is showing it is part of

<body class="hide-menu-for-landscape">

However when I view source of the HTML, "hide-menu-for-landscape" is just an empty iframe surrounded by script tags. I suspect the script is loading something into the iframe.

How would I get selenium to click on the "Show All" button at the bottom of the page?

Here's the code I have so far:

import time
from selenium import webdriver

sp500_url = "https://www.barchart.com/stocks/indices/sp/sp500?page=all"
driver = webdriver.Firefox(executable_path=r'D:\thirdparty\geckodriver-v0.27.0-win64\geckodriver.exe')

driver.get(sp500_url)
time.sleep(10)
driver.execute_script("window.scrollBy(1, 3000);")

# button = driver.find_elements_by_xpath("//a/u[contains(text(),'Show all')]")[0]
# button = driver.find_element_by_xpath("//a[@class='show-all']")
button = driver.find_element_by_css_selector('[class="show-all"]')
button.click()

html = driver.page_source
selenium python
1 Answers

I have used nearest HTML block "data-ng-if = 'Show all'" to click on the button

> <a data-ng-if="showAll" class="show-all" data-ng-click="getAll()">Showall</a>

import time
from selenium import webdriver

sp500_url = "https://www.barchart.com/stocks/indices/sp/sp500?page=all"
driver = webdriver.Chrome(executable_path=r'C:\Users\Raghul raj N\Downloads\chromedriver_win32\chromedriver.exe')

driver.get(sp500_url)
driver.execute_script("window.scrollBy(1, 3000);")

driver.find_element_by_css_selector("a[data-ng-if='showAll']").click()


html = driver.page_source
user_80071670
Related