how web scrap html table from money control website?

skye 0
Viewed 24

I am trying to get data from this website

But I am unable to get complete data from tables:

Here is my code:

import requests

from bs4 import BeautifulSoup

url = "https://www.moneycontrol.com/stocks/marketinfo/dividends_declared/"

r = requests.get(url)

soup = BeautifulSoup(r.text , 'html.parser')

dividendsTable = soup.find('table',class_ = 'b_12 dvdtbl')

data = []

import pandas

rows = dividendsTable.find_all('tr') 

for row in rows:

    st_company = row.find_all('td',class_ = "dvd_brdb")[0].text

    st_type = row.find_all('td',class_ = "dvd_brdb")[1].text

    st_percent = row.find_all('td',class_ = "dvd_brdb")[2].text

    st_announcement = row.find_all('td',class_ = "dvd_brdb")[3].text

    print(st_company,st_type,st_percent,st_announcement)

It returns list error and i am unable to change year to 2018.

beautifulsoup web-scraping python
1 Answers

To get the data of various years to pandas dataframe you can use next example:

import requests
import pandas as pd


url = "https://www.moneycontrol.com/stocks/marketinfo/dividends_declared/"
data = {"sel_year": "2018", "x": "1", "y": "1"}

all_dfs = []
for data["sel_year"] in range(2018, 2022 + 1):
    df = pd.read_html(requests.post(url, data=data).text)[1]
    df["Year"] = data["sel_year"]
    all_dfs.append(df)

df_out = pd.concat(all_dfs)
print(df_out.head(10))

Prints:

                 0         1         2             3           4            5  Year
0     COMPANY NAME  DIVIDEND  DIVIDEND          DATE        DATE         DATE  2018
1     COMPANY NAME      Type         %  Announcement      Record  Ex-Dividend  2018
2       Coal India   Interim     72.50    17-12-2018  31-12-2018   28-12-2018  2018
3            RITES   Interim     47.50    17-12-2018  28-12-2018   27-12-2018  2018
4              IOC   Interim     67.50    13-12-2018  25-12-2018   21-12-2018  2018
5  Dharamsi Morarj   Interim      5.00    07-12-2018  21-12-2018   20-12-2018  2018
6      Aarey Drugs   Interim      1.00    30-11-2018  14-12-2018   13-12-2018  2018
7           Nestle   Interim    500.00    28-11-2018  13-12-2018   12-12-2018  2018
8  Zodiac Ventures     Final      1.00    28-05-2018           -   11-12-2018  2018
9      BSE Limited   Interim    250.00    30-11-2018  12-12-2018   11-12-2018  2018
Andrej Kesely0
Related