Message Framing Method advantages and disadvantages in .NET

Question

I'm looking at the advantages and disadvantages for two the different TCP message framing methods I'm aware of.

Delimited: The TCP stream is broken into messages of a non fixed length by using a delimiter byte. When sending data the routine must check the message data for the delimiter and escape it to ensure safe transmission of the message frame. When receiving data the routine must read through the stream looking for the delimiter byte to break the frame into a message.

EG: User [Username] Password [Password]

Length-Prefixed: The TCP stream is broken into messages of a predetermined size by using a prefix of say 4 bytes to state the length of the message. When receiving data the routine will first read the prefix to determine the length of the message frame. When sending data the routine must prepend the length prefix to the message before transmission.

EG: [MessageLength]User [Username][MessageLength]Password [Password]

Both of these methods allow transmission of message frames that can vary in size and contain a stream of bytes to be interpreted. The higher level message structure or protocol is not relevant.

So I focus my attention towards scalability and performance efficiency. I find myself needing to run benchmark tests to see which method I can receive the greatest efficiency throughput without any message processing involved.

My current thoughts, I'm no expert by any means.

Delimited message frames would have lower efficiency during the receive routine as each byte in the stream needs to be checked for a message frame delimiter. Length-Prefixed message frames will always read the prefix bytes and the rest of the message frame stream will go straight into the buffer without processing until the entire message frame is received.

Where as Length-Prefixed message frames would have a lower efficiency during the send routine as the message prefix as to be transmitted before the message itself.

Other factors I can think could include:

Lots of small message frames would result in more packets being transmitted for a Length-Prefixed structure.
Lots of larger message frames would be process much more efficiently with a Length-Prefixed structure as each byte is not being read to check for a delimiter.

Any light across this topic would be awesome. I find it very hard to find good resources on the difference between Message Frame structures for TCP.

user_2648798 · Answer

I’ve tackled this problem a slightly different way before.

I take my initial payload (message) and Base64 encode it. This gives me an encoded string. I pick two characters that are known not be used in Base64. I use one for the start of my messages "*" and the other for the end "|".

Then I can just read the incoming bytes from stream until I get either my start or end character - and I know what to extract.

This is the code I wrote to do it:

/// 
/// Logic for converting to and from a 
/// 
public sealed class Packager : IPackager
{
    private static string START_DELIMITER => "*";
    private static string END_DELIMITER => "|";
    
    private static IEnumerable StartDelimiterBytes => Encoding.UTF8.GetBytes(START_DELIMITER);
    private static IEnumerable EndDelimiterBytes => Encoding.UTF8.GetBytes(END_DELIMITER);
    
    public byte[]? PacketToBytes(DataPacket? message)
    {
        if (message == null)
        {
            return null;
        }
    
        var returnBytes = new List();

        // START DELIMITER
        returnBytes.AddRange(StartDelimiterBytes);
    
        using (var ms = new MemoryStream())
        {
            using (var writer = new BsonDataWriter(ms))
            {
                new JsonSerializer().Serialize(writer, message);
            }
    
            // BODY BYTES
            var bodyAsString = Convert.ToBase64String(ms.ToArray());
            var bodyAsBytes = Encoding.UTF8.GetBytes(bodyAsString);
    
            returnBytes.AddRange(bodyAsBytes);
        }
   
        // END DELIMITER
        returnBytes.AddRange(EndDelimiterBytes);
    
        return returnBytes.ToArray();
    }
    
    public DataPacket? BytesToPacket(byte[]? bytes)
    {
        if (bytes == null)
        {
            return null;
                }
        var bytesAsString = Encoding.UTF8.GetString(bytes);
        if (string.IsNullOrEmpty(bytesAsString))
        {
            return null;
        }
    
        bytesAsString = bytesAsString.Replace(START_DELIMITER, string.Empty);
        bytesAsString = bytesAsString.Replace(END_DELIMITER, string.Empty);
    
        var decodedString = Convert.FromBase64String(bytesAsString);
    
        using (var ms = new MemoryStream(decodedString))
        {
            using (var reader = new BsonDataReader(ms))
            {
                return new JsonSerializer().Deserialize(reader);
            }
        }
     }
    
     public int GetFrameStartIndex(string value) => value.IndexOf(START_DELIMITER, StringComparison.InvariantCulture);
    
     public int GetFrameEndIndex(string value) => value.IndexOf(END_DELIMITER, StringComparison.InvariantCulture);
    
     public string ReadBytesAsString(byte[] value, int startIndex, int endIndex) => Encoding.UTF8.GetString(value, startIndex, endIndex);
    }

Message Framing Method advantages and disadvantages in .NET

2 Answers