Pyserial non-polling read

I am using Pyserial(a Python Serial lib) to read hex data from serial port on a raspberry pi, OS is raspbian.

I need to read from serial port one byte each time, and process the data immediately without any time delay. The way I am doing it now is just polling, constantly read from the port within in a loop. The problem is that it's taking almost 95% of CPU, and make other process slow.

I believe there is a better way to do this without using too much CPU resource. I used 'minicom' to monitor serial port, and it's not taking much CPU. So I wonder anyone who is familiar with Pyserial could help me, or anyone can tell me how minicom has done it.

Here is my code. Thanks a lot

while True:
    try:
        reading = ser.read().encode('hex')

        if reading == "":
            continue

        #always add to packet when it's not empty
        data_packet.append(reading)

        if check_byte == True:
            #sometimes, check byte could be 2a
            check_byte = False
            continue
        elif length_byte == True:
            #this byte is length
            packet_length = int(reading, 16)
            length_byte = False

        if packet_length == 0 and hex_equals(reading, "2a"):
            #start of a string
            #next byte is length
            length_byte = True
        elif packet_length > 0:
            packet_length -= 1
        elif hex_equals(reading, "23"):
            #end of packet
            data_packet_handle = data_packet
            data_packet = []
            handle_data(data_packet_handle)
        elif packet_length == 0:
            #if it's zero, the packet is over, but still have check and ending character
            #next byte is check byte
            check_byte = True
    except Exception, err:
        pass
        print Exception, err