How can I encode html file after read file with ZipFile?

user_11651988 0
Viewed 145

I am reading a zip file from a URL. Inside the zip file, there is an HTML file. After I read the file everything works fine. But when I print the text I am facing a Unicode problem. Python version: 3.8

from zipfile import ZipFile
from io import BytesIO
from bs4 import BeautifulSoup
from lxml import html
content = requests.get("www.url.com")
zf = ZipFile(BytesIO(content.content))
file_name = zf.namelist()[0]
file = zf.open(file_name)

soup = BeautifulSoup(file.read(),'html.parser',from_encoding='utf-8',exclude_encodings='utf-8')
for product in soup.find_all('tr'):
    product = product.find_all('td')
    if len(product) < 2: continue
    print(product[1].text)

I already try to open file and print text with .decode('utf-8') I got following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 0: invalid continuation byte

I add from_encoding and exclude_encodings in BeautifulSoup but nothing change and I didn't get an error.

Expected prints:

ÇEŞİTLİ MADDELER TOPLAMI
Tarçın
Fidanı

What I am getting:

ÇEÞÝTLÝ MADDELER TOPLAMI
Tarçýn
Fidaný
beautifulsoup unicode character-encoding python
1 Answers

I look at the file and the encoding is not utf-8, but iso-8859-9. Change the encoding and everything will be fine:

soup = BeautifulSoup(file.read(),'html.parser',from_encoding='iso-8859-9')

This will output: ÇEŞİTLİ MADDELER TOPLAMI

user_39543790
Related