Python decode() 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

user_8648894 0
Viewed 3641

I am building this socket application and every time I am getting the following error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Here are the relevant lines from the server:

    filename = client_sock.recv(1024).decode()
    filesize = client_sock.recv(1024).decode()

Here are the relevant lines from the client:

    self.sock.send(file_dir.encode())
    self.sock.send(str(filesize).encode())

The error message happens on the second line of the server. (filesize =) Here are prints showing what the client will send to the server.

    print(file_dir) # Output is D:/Statispic2/Photos/photo3.jpg
    print(filesize) # Output is 96523

This error only happens sometimes which is really weird. I have looked at other questions that asked a similar question but their solutions either didn't work or weren't relevant.

If you want to see the whole code or have any more questions please let me know! Thanks a lot!

tcp sockets python
3 Answers

The errors happen because that byte cannot be decoded to utf-8, you can handle that as exception in, with decode it as 'utf-16' on exception handling:

filename = client_sock.recv(1024)
filesize = client_sock.recv(1024)
try:
    decoded_filename = filename.decode()
    decoded_filesize = filename.decode()
except UnicodeDecodeError:
    decoded_filename = filename.decode('utf-16')
    decoded_filesize = filename.decode('utf-16')

Alternatively, you can ignore exceptions during decoding, but it's not recommended way...

filename = client_sock.recv(1024).decode("utf-8", "ignore")
filesize = client_sock.recv(1024).decode("utf-8", "ignore")
user_20679760

Your method of reading in data is unreliable, so you're probably reading more data than you want in the first recv and getting non-character data in the second receive. TCP is a streaming protocol, just because you send two strings in separate send() calls does not mean the receiver will get them in separate recv() calls.

If you are sending distinct pieces of data you must have some method of knowing when one piece ends and the next piece begins. The rules for your solution constitute a protocol.

See also my answer and others for reliably reading in exactly K bytes. This may be useful if your protocol prefixes each piece with the length of that piece.

user_2387040

I've had similar issue. I just removed .decode() and I got appropirate results, so in your case:

    self.sock.send(file_dir)
    self.sock.send(str(filesize))
user_73469130
Related