Python ASCII and Unicode decode error

user_1484057 0
Viewed 57866

I got this very very frustrating error when inserting a certain string into my database. It said something like:

Python cannot decode byte characters, expecting unicode"

After a lot of searching, I saw that I could overcome this error by encoding my string into Unicode. I try to do this by decoding the string first and then encoding it in UTF-8 format. Like:

string = string.encode("utf8")

And I get the following error:

'ascii' codec can't decode byte 0xe3 in position 6: ordinal not in range(128)

I have been dying with this error! How do I fix it?

character-encoding sqlite python string
5 Answers

string in python 2.7 is an ecoded string (encoded in ASCII mostly) but not a character string or unicode.

So when you do string.encode('some encoding') you are actually encoding an encoded string (using some encoding)

Python has to first decode that string using default encoding (ASCII in python 2.7) and then it will further encode. Your string is not encoded in ASCII but some other encoding (UTF8, LATIN-1..), so when python tries to decode this using ASCII, it throws an error because ASCII codec cannot decode few characters in your given string which are out of ASCII range (0 - 127)

#to encode above given string, first decode that using some encoding
decoded_string = string.decode('utf8')
#now encode that decoded string
decoded_string.encode('utf8')
user_48132070
Related