Un-defined behavior when using hex values without Casting

Question

This is the strangest problem I have ever encountered. I am programming a chess engine and I have a function that returns a rank mask. For example, Consider this 64-bit bitboard.

The 1 in the middle is a piece.

Now my function should return

Very simple, just return a mask of the rank. Here is my code

uint64_t get_rank_mask(int sq)
{
    return (0xFF<<(sq-(sq%8)));
}

Here is what the function returns when I pass 36, or the square e5

Hmm, strange it is. Let me make a small change in the code.

uint64_t get_rank_mask(int sq)
{
    uint64_t mask = 0xFF;
    mask<<=(sq-(sq%8));
    return  mask;
}

Here is the return value now

And magically, that is the intended result. I don't even know the difference between the two code samples. Can anyone explain this behaviour?

user_8372853 · Answer

0xFF is an int. Assuming an int has 32 bits, shifting this value left by 32 will invoke undefined behavior.

In the second version, you are storing 0xFF in an unsigned integral type. This can be shifted any number of bits, and that operation is well defined. In your case, you are storing it in an unsigned int with 64 bits, and so shifting left by 32 gives the result you want.

Un-defined behavior when using hex values without Casting

1 Answers