I have created a program that collects table data at the following location. And when extracting data in soup library, it appears fine, but when converting html codes to a table using pandas library pd.read_html(table) I get an error message, I don't know why
this code bellow :
import requests
from bs4 import BeautifulSoup
import pandas as pd
header = {
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36'
}
url = "https://www.worldometers.info/coronavirus/#countries"
req = requests.get(url, headers=header)
#test of response server (permession for client)
""" True response with 200 and not forbiden access """
#soup methode : extract data finded in html page from the link
soup = BeautifulSoup(req.content, 'lxml')
tables = soup.find('table',{'id':'main_table_countries_today'})
df = pd.read_html(tables)
print(df)
after excute :
Traceback (most recent call last):
File "c:\Users\pc\Desktop\manual scrape\scrape 1\covid find all data.py", line 18, in <module>
df = pd.read_html(tables)
File "C:\Users\pc\AppData\Roaming\Python\Python39\site-packages\pandas\util\_decorators.py", line 299, in wrapper
return func(*args, **kwargs)
File "C:\Users\pc\AppData\Roaming\Python\Python39\site-packages\pandas\io\html.py", line 1085, in read_html
return _parse(
File "C:\Users\pc\AppData\Roaming\Python\Python39\site-packages\pandas\io\html.py", line 893, in _parse
tables = p.parse_tables()
File "C:\Users\pc\AppData\Roaming\Python\Python39\site-packages\pandas\io\html.py", line 213, in parse_tables
tables = self._parse_tables(self._build_doc(), self.match, self.attrs)
File "C:\Users\pc\AppData\Roaming\Python\Python39\site-packages\pandas\io\html.py", line 717, in _build_doc
r = parse(self.io, parser=parser)
File "C:\Users\pc\AppData\Roaming\Python\Python39\site-packages\lxml\html\__init__.py", line 939, in parse
return etree.parse(filename_or_url, parser, base_url=base_url, **kw)
File "src\lxml\etree.pyx", line 3521, in lxml.etree.parse
File "src\lxml\parser.pxi", line 1875, in lxml.etree._parseDocument
TypeError: 'NoneType' object is not callable
PS C:\Users\pc\Desktop\manual scrape\scrape 1>
The program aims to print table data at code execution time using pd.read_html()
Such as :
