Scrape data from csv downloaded after right clicking on webpage using selenium python

user_11896115 0
Viewed 349

I'm looking to scrape data from a webpage using python and selenium. There is a csv download option which is visible only after a right click in the frame of the graph. I am not able to right click on the page and click on csv - download option using selenium. Here is the link for web page from where I am trying to get data - https://datastudio.google.com/reporting/d97f5736-2b85-4f39-beba-6dc386c24429/page/Z3ToB Have tried following set of code to do that:

options = webdriver.ChromeOptions()
    options.binary_location = r"<Path where chrome application is installed>"
    driver = webdriver.Chrome(r"<path to chrome driver>",chrome_options=options)
    driver.get("https://datastudio.google.com/reporting/d97f5736-2b85-4f39-beba-6dc386c24429/page/Z3ToB")
    timeout = 10
    from selenium.webdriver import ActionChains
    action = ActionChains(driver)
    action.move_to_element(driver.find_element_by_xpath("//lego-canvas-container[@class='lego-canvas-container']")).perform()
    action.context_click().perform()

Using this, not able to find the given XPATH, even tried with class name like report area. Could anyone guide about how to right click on anywhere in the frame and then find the download csv option within that?

webdriverwait webdriver selenium python
4 Answers

Because of javascript going to visible after right click its unable to find xpath without right click try this code its worked on me

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
chrome_options.add_argument("disable-infobars")
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('--disable-blink-features=AutomationControlled') 
driver = webdriver.Chrome(executable_path = 'chromedriver.exe',options = chrome_options)
driver.implicitly_wait(10)
driver.get("https://datastudio.google.com/reporting/d97f5736-2b85-4f39-beba-6dc386c24429/page/Z3ToB")
action = ActionChains(driver)
action.pause(1)
action.move_by_offset(150,150).perform()
action.context_click().perform()
action.move_to_element(driver.find_element_by_xpath('//*[@id="mat-menu-panel-0"]/div/span[5]/button')).perform()
action.click().perform()
user_148642370

Use the below xpath to identify the element and then right click and then find the csv button and click.

driver.get("https://datastudio.google.com/reporting/d97f5736-2b85-4f39-beba-6dc386c24429/page/Z3ToB")
time.sleep(5) #delay to load page properly. you can use explicit wait as well
element=driver.find_element_by_xpath("//div[@class='drop-zone-text']")
action = ActionChains(driver)
action.move_to_element(element).perform()
action.context_click().perform()
#To click on download csv
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.XPATH,"//button[contains(.,'Download CSV')]"))).click()

You need to import below libraries

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains

Browser snapshot:

enter image description here

user_108856840

I found it a bit easier to right click and then use the arrow keys to select the appropriate option.

So you can perform a right click/context_click anywhere on the canvas, to open the menu popup. Then you can move up and down with arrow keys and select the 'Download Csv' option.

actions = ActionChains(driver)

# Find the canvas element
element = driver.find_element_by_xpath('//*[@id="body"]/div/div/div[1]/div[2]/div/div[1]/div[1]/div[1]/div/lego-report/lego-canvas-container/div/file-drop-zone/span/content-section/div[3]/canvas-component')

# Right click the element, then press the Down key twice followed by the Enter to move to the Download CSV option and select it.
actions.move_to_element(element).context_click().send_keys([Keys.DOWN, Keys.DOWN, Keys.ENTER]).perform()
user_141573380 
driver.get("https://datastudio.google.com/reporting/d97f5736-2b85-4f39-beba-6dc386c24429/page/Z3ToB")
time.sleep(5)
source= wait.until(EC.presence_of_element_located((By.XPATH,"/html/body")))
action = ActionChains(driver)
action.context_click(source).perform()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#mat-menu-panel-0 > div > span:nth-child(5) > button"))).click()

Weirdly got it to work with this. Seems you need to wait and the context click the body and then click on the menu element.

<button _ngcontent-fys-c1="" class="mat-focus-indicator mat-tooltip-trigger mat-menu-item ng-star-inserted" mat-menu-item="" role="menuitem" tabindex="0" aria-disabled="false"> Download CSV <!----><!----><!----><div class="mat-menu-ripple mat-ripple" matripple=""></div></button>

Import

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
user_99012610
Related