Why does BitConverter seemingly return incorrect results when converting floats and bytes?

user_7281613 0
Viewed 399

I'm working in C# and attempting to pack four bytes into a float (the context is game development, where an RGBA color is packed into a single value). To do this, I'm using BitConverter, but certain conversions seem to result in incorrect bytes. Take the following example (using bytes 0, 0, 129, 255):

var before = new [] { (byte)0, (byte)0, (byte)129, (byte)255 };
var f = BitConverter.ToSingle(before, 0); // Results in NaN
var after = BitConverter.GetBytes(f); // Results in bytes 0, 0, 193, 255

Using https://www.h-schmidt.net/FloatConverter/IEEE754.html, I verified that the four bytes I started with (0, 0, 129, 255, equivalent to binary 00000000000000001000000111111111) represents the floating-point value 4.66338115943e-41. By flipping the endianness (binary 11111111100000010000000000000000), I get NaN (which matches f in the code above). But when I convert that float back to bytes, I get 0, 0, 193, 255 (note 193 when I'm expecting 129).

Curiously, running this same example with bytes 0, 0, 128, 255 is correct (the floating-point value f becomes -Infinity, then converting back to bytes yields 0, 0, 128, 255 again). Given this fact, I suspect NaN is relevant.

Can anyone shed some light on what's happening here?

Update: the question Converting 2 bytes to Short in C# was listed as a duplicate, but that's inaccurate. That question is attempting to convert bytes to a value (in that case, two bytes to a short) and incorrect endianness was giving an unexpected value. In my case, the actual float value is irrelevant (since I'm not using the converted value as a float). Instead, I'm attempting to effectively reinterpret four bytes as a float directly by first converting to a float, then converting back. As shown, that back-and-forth sometimes returns different bytes than the ones I sent in.

Second update: I'll simply my question. As Peter Duniho comments, BitConverter will never modify the bytes you pass in, but simply copy them to a new memory location and reinterpret the result. However, as my example shows, it is possible to send in four bytes (0, 0, 129, 255) which are internally copied and reinterpreted to a float, then convert that float back to bytes that are different than the originals (0, 0, 193, 255).

Endianness is frequently mentioned in relation to BitConverter. However, in this case, I feel endianness isn't the root issue. When I call BitConverter.ToSingle, I pass in an array of four bytes. Those bytes represent some binary (32 bits) which is converted to a float. By changing the endianness prior to the function call, all I'm doing is changing the bits I send into the function. Regardless of the value of those bits, it should be possible to convert them to a float (also 32 bits), then convert the float back to get the same bits I sent in. As demonstrated in my example, using bytes 0, 0, 129, 255 (binary 00000000000000001000000111111111) results in a floating-point value. I'd like to take that value (the float represented by those bits) and convert it to the original four bytes.

Is this possible in C# in all cases?

bitconverter endianness c#
1 Answers

After research, experimentation, and discussion with friends, the root cause of this behavior (bytes changing when converted to and from a float) seems to be signaling vs. quiet NaNs (as Hans Passant also pointed out in a comment). I'm no expert on signaling and quiet NaNs, but from what I understand, quiet NaNs have the highest-order bit of the mantissa set to one, while signaling NaNs have that bit set to zero. See the following image (taken from https://www.h-schmidt.net/FloatConverter/IEEE754.html) for reference. I've drawn four colored boxes around each group of eight bits, as well as an arrow pointing to the highest-order mantissa bit.

Visual representation of a float's bit layout.

Of course, the question I posted wasn't about floating-point bit layout or signaling vs. quiet NaNs, but simply asking why my encoded bytes were seemingly modified. The answer is that the C# runtime (or at least I assume it's the C# runtime) internally converts all signaling NaNs to quiet, meaning that the byte encoded at that position has its second bit swapped from zero to one.

For example, the bytes 0, 0, 129, 255 (encoded in the reverse order, I think due to endianness) puts the value 129 in the second byte (the green box). 129 in binary is 10000001, so flipping its second bit gives 11000001, which is 193 (exactly what I saw in my original example). This same pattern (the encoded byte having its value changed) applies to all bytes in the range 129-191 inclusive. Bytes 128 and lower aren't NaNs, while bytes 192 and higher are NaNs, but don't have their value modified because their second bit (placed at the highest-order mantissa bit) is already one.

So that answers why this behavior occurs, but in my mind, there are two questions remaining:

  1. Is it possible to disable this behavior (converting signaling NaNs to quiet) in C#?
  2. If not, what's the workaround?

The answer to the first question seems to be no (I'll amend this answer if I learn otherwise). However, it's important to note that this behavior doesn't appear consistent across all .NET versions. On my computer, NaNs are converted (i.e. my encoded bytes changed) on every .NET Framework version I tried (starting with 4.8.0, then working back down). NaNs appear to not be converted (i.e. my encoded bytes did not change) in .NET Core 3 and .NET 5 (I didn't test every available version). In addition, a friend was able to run the same sample code on .NET Framework 4.7.2, and surprisingly, the bytes were not modified on his machine. The internals of different C# runtimes isn't my area of expertise, but suffice to say there's variance among versions and computers.

The answer to the second question is to, as others have suggested, simply avoid the float conversion entirely. Instead, each set of four bytes (representing RGBA colors in my case) can either be encoded in an integer or added to a byte array directly.

user_72816130
Related