web scraping - refer to element

user_16305517 0
Viewed 29

I need to get name of the bank and the sum to the pandas dataframe as two columns

from bs4 import BeautifulSoup
import requests
import html5lib
import pandas as pd

html_data = requests.get('https://en.wikipedia.org/wiki/List_of_largest_banks', timeout=20).content

soup = BeautifulSoup(html_data, "html.parser")

newDataFrame = pd.DataFrame(columns=['Name', 'Market Cap (US$ Billion)'])
for row in soup.find_all('tbody')[3].find_all('tr'):
    col = row.find_all('td')
    print(col)

so the first row is :

[
<td>1</td>, 
<td><a href="/wiki/JPMorgan_Chase" title="JPMorgan Chase">JPMorgan Chase</a></td>, 
<td>368.78</td>
]

how can i refer to the td inside td? If i want to get bank's name inside td for ex

pandas web-scraping beautifulsoup html python
1 Answers

If you need Dataframe from the wikipedia article, simply use pd.read_html function:

import pandas as pd

df = pd.read_html("https://en.wikipedia.org/wiki/List_of_largest_banks")[3]
print(df.head().to_markdown())

Prints:

Rank Bank name Market cap(US$ billion)
0 1 JPMorgan Chase 368.78
1 2 Industrial and Commercial Bank of China 295.65
2 3 Bank of America 279.73
3 4 Wells Fargo 214.34
4 5 China Construction Bank 207.98
user_100359850
Related