How can I scrape "Net Worth" from YieldWatch?

user_1583516 0
Viewed 430

I would to scrape a website as per following:

  1. Go to Yieldwatch
  2. Enter 0xF54274757Bf717B1ab52bA0d3a7CbF635f856a0d into the Address textbox and click the binoculars
  3. Scrape the text under "Net worth" enter image description here

I am using Selenium in Python to try and do it and here is my attempt so far:

from selenium.webdriver import Safari  # pip install selenium
from selenium.webdriver.support.ui import WebDriverWait

url = 'https://www.yieldwatch.net/'
bsc_public_key = '0xF54274757Bf717B1ab52bA0d3a7CbF635f856a0d'

# with closing(Safari()) as browser:
browser = Safari()
browser.get(url)

textbox = browser.find_element_by_id('addressInputField')
textbox.clear()
textbox.send_keys(bsc_public_key)

button = browser.find_element_by_class_name('binoculars icon')
button.click()

# # wait for the page to load
WebDriverWait(browser, timeout=20).until(
    lambda x: x.find_element_by_id('ui centered image'))

# store it to string variable
page_source = browser.page_source
print(page_source)

The code doesn't work. After the browser loads, I don't see the textbox filled in with the address. How can I do this in Python (with or without Selenium)?

web-scraping selenium python
2 Answers

You don't really need the heavy guns of selenium. You can get all the data from the API endpoint.

Here's how:

import requests

wallet = "0xF54274757Bf717B1ab52bA0d3a7CbF635f856a0d"
endpoint = f"https://www.yieldwatch.net/api/all/{wallet}?platforms=beefy,pancake,hyperjump,auto,mdex"

wallet_data = requests.get(endpoint).json()["result"]

# https://yieldwatch.medium.com/yieldwatch-faqs-93c2cde244bf
# Net Value = Total Deposit + Total Yield + Wallet Balance — Total Debt
total = sum(v["totalUSDValues"]["total"] for v in wallet_data["PancakeSwap"].values())
net_worth = total + wallet_data["walletBalance"]["totalUSDValue"]
print(f"Net worth for {wallet}:\n{round(net_worth, 2)}")

This outputs most up-to-date net worth:

Net worth for 0xF54274757Bf717B1ab52bA0d3a7CbF635f856a0d:
2213.13

Note: The FAQ says you need Total Debt to calculate the net worth but this wallet doesn't have any debt so I didn't include the value in the equation.

However, if you have a wallet that has a debt, please share it, and I'll update the answer.

user_61067910

You are trying to read the ui centered image element text when it is still not presenting the final value.
Try waiting for invisibility_of_element_located(By.XPATH,"//div[@class='content']//div[@class='header']")
And only after that read the text from ui centered image element.

user_34854340
Related