Retrieving text from Python Selenium

user_3114229 0
Viewed 66

I'm trying to pull some Diablo II trading prices of a trading page using Selenium.

So far I've managed to locate the object I'm interested in using classes, but I can't retrieve the actual text which is what I need.

I have the following code:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By

driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get('https://traderie.com/diablo2resurrected/product/3382986705/buying?prop_Platform=PC&prop_Mode=softcore&makeOffer=false')

# Close cookie popup
driver.find_element(By.XPATH, '//*[@id="tyche_cmp_modal"]/div/div/div/div[5]/div[2]/a').click()

# Locate entire page of offers
All = driver.find_element(By.CLASS_NAME, "row")

# Locate individual offer
Offer = All.find_element(By.CLASS_NAME, "col-xs-12")

# Locate price in each offer
Price = Offer.find_element(By.CLASS_NAME, "sc-eCImPb")

# Print price
print(str(Price.text))

# Close page
driver.close()

The print turns out empty, but it should return something like 3x Vex, or similar. What am I doing wrong?

selenium python
1 Answers

There are several issues here:

  1. You should add waits. Preferably Expected Conditions explicit waits.
  2. You are using a wrong locator for price element
  3. Since there are multiple offers there you should iterate over the results in a loop.
  4. Variable names should be lowercased according to accepted convention.

I think your code should be something like this:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
wait = WebDriverWait(driver, 20)

driver.get('https://traderie.com/diablo2resurrected/product/3382986705/buying?prop_Platform=PC&prop_Mode=softcore&makeOffer=false')

# Close cookie popup
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="tyche_cmp_modal"]/div/div/div/div[5]/div[2]/a'))).click()

wait.until(EC.visibility_of_element_located((By.XPATH, "//a[contains(@style,'capitalize')]")))
time.sleep(1)

prices = driver.find_elements(By.XPATH, "//a[contains(@style,'capitalize')]")

for price in prices:
    print(price.text)

# Close page
driver.close()

UPD
To iterate over offers and get each offer seller's name and the price you can do something like this:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
wait = WebDriverWait(driver, 20)

driver.get('https://traderie.com/diablo2resurrected/product/3382986705/buying?prop_Platform=PC&prop_Mode=softcore&makeOffer=false')

# Close cookie popup
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="tyche_cmp_modal"]/div/div/div/div[5]/div[2]/a'))).click()

wait.until(EC.visibility_of_element_located((By.XPATH, "//a[contains(@style,'capitalize')]")))
time.sleep(1)

offers = driver.find_elements(By.CLASS_NAME, "col-xs-12")

for offer in offers:
    name = offer.find_element(By.XPATH, ".//a[contains(@href,'profile')]")
    prices = offer.find_elements(By.XPATH, ".//a[contains(@style,'capitalize')]")
    #now you can extract texts from name with name.text
    #and iterate over prices with
    for price in prices:
        price_text = price.text
    #You can put all these in dictionary etc.


# Close page
driver.close()
user_34854340
Related