x64 argument and return value calling convention

Question

I invoke Clang 12.0.0 with -Os -march=haswell to compile the following C program:

int bar(int);

int foo(int x) {
  const int b = bar(x);
  if (x || b) {
      return 123;
  }
  return 456;
}

The following assembly is generated:

foo:                                    # @foo
        push    rbx
        mov     ebx, edi
        call    bar
        or      eax, ebx
        mov     ecx, 456
        mov     eax, 123
        cmove   eax, ecx
        pop     rbx
        ret

https://gcc.godbolt.org/z/WsGoM56Ez

As I understand it, the caller of foo sets up x in RAX/EAX. foo then calls bar, which doesn't require modifying RAX/EAX, since x is passed through as unmodified input.

The or eax, ebx instruction appears to be comparing the input x with the result of bar. How does that result end up in EBX? What purpose does mov ebx,edi serve?

user_4593267 · Answer

I'm afraid you are mistaken:

the function argument is passed in rdi, as per the x86-64 System V calling convention.
register rbx must not be modified by a function; GCC saves/restores it as required, so it can keep a copy of x there across the call to bar.
the function return value is in rax. (Actually eax; a 32-bit int only uses the low half)

You can verify the basics by compiling a function like int foo(int x){return x;} - you'll see just a mov eax, edi.

Here is a commented version:

foo:                                    # @foo
        push    rbx           # save register rbx
        mov     ebx, edi      # save argument `x` in ebx
        call    bar           # a = bar()  (in eax)
        or      eax, ebx      # compute `x | a`, setting FLAGS
        mov     ecx, 456      # prepare 456 for conditional move
        mov     eax, 123      # eax = 123
        cmove   eax, ecx      # if `(x | a) == 0` set eax to 456
        pop     rbx           # restore register rbx
        ret                   # return value is in eax

The compiler optimizes x || b as (x | b) != 0 which allows for branchless code generation.

Note that mov doesn't modify the FLAGS, unlike most integer ALU instructions.

x64 argument and return value calling convention

1 Answers