How to scrape the yahoo earnings calendar with beautifulsoup

user_5828859 0
Viewed 1021

How can I scrape the yahoo earnings calendar to pull out the dates?

This is for python 3.

from bs4 import BeautifulSoup as soup
import urllib

url = 'https://finance.yahoo.com/calendar/earnings?day=2019-06-13&symbol=ibm'

response = urllib.request.urlopen(url)
html = response.read()

page_soup = soup(html,'lxml')
table = page_soup.find('p')
print(table)

the output is "None"

web-scraping beautifulsoup html python
3 Answers

Beautiful Soup has some find functions that you can use to inspect the DOM , please refer to the documentation

from bs4 import BeautifulSoup as soup
import urllib.request

url = 'https://finance.yahoo.com/calendar/earnings?day=2019-06-13&symbol=ibm'

response = urllib.request.urlopen(url)
html = response.read()

page_soup = soup(html,'lxml')
table = page_soup.find_all('td')
Dates = []
for something in table:
    try:
        if something['aria-label'] == "Earnings Date":
            Dates.append(something.text)
    except:
        print('')

print(Dates)
user_72231840

Might be off-topic but since you want to get a table from a webpage, you might consider using pandas which works with two lines:

import pandas as pd
earnings = pd.read_html('https://finance.yahoo.com/calendar/earnings?day=2019-06-13&symbol=ibm')[0]
user_76525440

Here are two succinct ways 

import requests
from bs4 import BeautifulSoup as bs

r = requests.get('https://finance.yahoo.com/calendar/earnings?day=2019-06-13&symbol=ibm&guccounter=1')
soup = bs(r.content, 'lxml')
# using attribute = value selector
dates = [td.text for td in soup.select('[aria-label="Earnings Date"]')]
#using nth-of-type to get column
dates = [td.text for td in soup.select('#cal-res-table td:nth-of-type(3)')]
user_62412350
Related