Bytes operations in Python

user_13101739 0
Viewed 398

I'm working on a project in which I have to perform some byte operations using python and I'd like to understand some basic principals before I go on with it.

t1 = b"\xAC\x42\x4C\x45\x54\x43\x48\x49\x4E\x47\x4C\x45\x59"
t2 = "\xAC\x42\x4C\x45\x54\x43\x48\x49\x4E\x47\x4C\x45\x59"
print("Adding b character before: ",t1)
print("Using bytes(str): ",bytes(t2,"utf-8"))
print("Using str.encode: ",t2.encode())

In particular, I cannot understand why the console prints this when I run the code above:

C:\Users\Marco\PycharmProjects\codeTest\venv\Scripts\python.exe C:/Users/Marco/PycharmProjects/codeTest/msgPack/temp.py
Adding b character before:  b'\xacBLETCHINGLEY'
Using bytes(str):  b'\xc2\xacBLETCHINGLEY'
Using str.encode:  b'\xc2\xacBLETCHINGLEY'

What I would like to understand is why, if I use bytes() or decode, I get an extra "\xc2" in front of the value. What does it mean? Is this supposed to appear? And if so, how can I get rid of it without using the first method?

decode encode python-3.x byte python
2 Answers

Because bytes objects and str objects are two different things. The former represents a sequence of bytes, the latter represents a sequence of unicode code points. There's a huge difference between the byte 172 and the unicode code point 172.

In particular, the byte 172 doesn't encode anything in particular in unicode. On the other hand, unicode code point 172 refers to the following character:

>>> c = chr(172)
>>> print(c)
¬

And of course, they actual raw bytes this would correspond to depend on the encoding. Using utf-8 it is a two-byte encoding:

>>> c.encode()
b'\xc2\xac'

In the latin-1 encoding, it is a 1 byte:

>>> c.encode('latin')
b'\xac'

If you want raw bytes, the most precise/easy way then is to use a bytes-literal.

user_50144550

In a string literal, \xhh (h being a hex digit) selects the corresponding unicode character U+0000 to U+00FF, with U+00AC being the ¬ "not sign". When encoding to utf-8, all code points above 0x7F take two or more bytes. \xc2\xac is the utf-8 encoding of U+00AC.

>>> "\u00AC" == "\xAC"
True
>>> "\u00AC" == "¬"
True
>>> "\xAC" == "¬"
True
>>> "\u00AC".encode('utf-8')
b'\xc2\xac'
>>> "¬".encode("utf-8")
b'\xc2\xac'
user_6420700
Related