Unable to find a 'table' while the table is showing on the page

user_6387095 0
Viewed 54

I am trying to scrape a page.

HTML element:

<table id="stock_ret_data" class="table table-bordered 
table-striped table-sm sortable-theme-bootstrap dataTable 
no-footer" data-sortable="" role="grid" 
aria-describedby="stock_ret_data_info" style="width: 1115px;">

When I try to use beautifulsoup to find the table using:

soup = bs(requests.get(url).content, 'html.parser')
tbl = soup.find('table',{'id':'stock_ret_data'})

It returns empty. While the same code works for another table on the same page.

I cannot figure out what I might be doing wrong.

beautifulsoup python
1 Answers

The page loads the data from this Json URL:

import json
import requests


url = 'https://www.rupeevest.com/mf_stock_portfolio/get_stock_detail?fincode=100002'
data = requests.get(url).json()

# uncomment this to see all data:
# print(json.dumps(data, indent=4))

for d in data['stock_data']:
    print(json.dumps(d, indent=4))
    print('-' * 80)

Prints:

{
    "s_name1": "ITI Balanced Advantage Fund-Reg(G)",
    "fund_manager_code": 541,
    "fund_house": "ITI Mutual Fund",
    "rv_sect_name": "Capital Goods",
    "fund_mgr1": "George Heber Joseph",
    "fincode": 100002,
    "compname": "ABB India Limited",
    "aum": null,
    "percent_aum": null,
    "month_name_1": null,
    "schemecode": 44362,
    "primary_fd_code": 44362,
    "month_name_2": null,
    "month_name_3": null,
    "month_name_4": 33547
}
--------------------------------------------------------------------------------
{
    "s_name1": "ITI Long Term Equity Fund-Reg(G)",
    "fund_manager_code": 541,
    "fund_house": "ITI Mutual Fund",
    "rv_sect_name": "Capital Goods",
    "fund_mgr1": "George Heber Joseph",
    "fincode": 100002,
    "compname": "ABB India Limited",
    "aum": null,
    "percent_aum": null,
    "month_name_1": null,
    "schemecode": 44014,
    "primary_fd_code": 44014,
    "month_name_2": null,
    "month_name_3": 1925,
    "month_name_4": 1925
}
--------------------------------------------------------------------------------

...and so on.
user_100359850
Related