Facing problem scraping bangla text with python3

user_9785433 0
Viewed 82

I am scraping Bangla text from This Website. It was working fine earlier but suddenly it is showing some random text.

Code:

from urllib.request import urlopen,Request
from lxml import html as  h
req = Request('https://corona.gov.bd', headers={'User-Agent': 'Mozilla/5.0','Content-type': 'text/plain; charset=utf-8'})
html = urlopen(req)
tree = h.fromstring(html.read())

new_infected = tree.xpath(
                '/html/body/section[3]/div/div[1]/div[1]/div/div[1]/div[1]/h3/b')[0].text
print(new_infected)

Output:

১৬৯৪

Expected Output:

১৬৯৪
web-scraping python-3.x
1 Answers

You can create a new HTMLParser specifying the encoding :

parser = lxml.etree.HTMLParser(encoding = 'utf-8')
tree = h.fromstring(html.read(), parser = parser)

For example :

from urllib.request import urlopen,Request
from lxml import html as  h
import lxml
req = Request('https://corona.gov.bd', headers={'User-Agent': 'Mozilla/5.0','Content-type': 'text/plain; charset=utf-8'})
html = urlopen(req)
parser = lxml.etree.HTMLParser(encoding = 'utf-8')
tree = h.fromstring(html.read(), parser = parser)

new_infected = tree.xpath('/html/body/section[3]/div/div[1]/div[1]/div/div[1]/div[1]/h3/b')[0].text
print(new_infected)
user_26143640
Related