Only the low half of EAX is zeroed? (MASM using .486 / .model flat,stdcall)

user_4463557 0
Viewed 132

Running the code below in VS2019, I was surprised with the fact that the register EAX is not zeroed with the instruction xor eax, eax.

_text SEGMENT
.486
.model flat,stdcall
.stack 4096

.code
main PROC
    xor eax, eax    
    ret
main ENDP
_text ENDS
END

P.S.: I was expecting the EAX register to be zero after the instruction xor eax, eax. Debugging the code with a break point in the ret instruction I can see in the Registers window that only AX is zeroed. Nothing happens to the bits 16-31 in EAX. In x64 the instruction xor eax, eax executes correctly, zeroing the entire RAX register as expected.

masm x86 assembly
1 Answers

Based on your comments, it seems that the SEGMENT directive somehow convinced the assembler that it was assembling code to be run in 16-bit mode.

In 16-bit mode, instructions operating on 32-bit registers like eax need a 66 operand-size prefix, so the assembler emitted 66 33 C0. However, the program actually ran in 32-bit mode, in which the sense of the 66 operand-size prefix is reversed: 32-bit operands are the default and 66 selects 16-bit operands. So when the program ran, the instruction executed as xor ax, ax and didn't zero the high bits of eax.

You said that removing the SEGMENT directive fixed it. I don't know exactly why it had this effect, but other examples of 32-bit MASM programs don't include this directive anyway, so it should probably never have been there at all.

user_6349190
Related