Compiling an OS and defining the system calls

Viewed 30

I'm trying to better understand operating systems, not the theory behind them but how real people write real OS code.

I know most OS's are written in C. I know the source code for these OS's include calls to functions like malloc, calloc, etc, to allocate memory for a process, etc.

Under normal conditions, i.e, when compiling code destined to run on an OS, I know that the C compiler will use the underlying OS's system calls to execute these functions. But when compiling the source code for these OS's, how does the compiler know what to do. The system calls don't exist cause they're defined by the OS. Does the compiler just call some assembly routine, which will eventually become a system call?

1 Answers

It is complex because you need to understand several things about OS development to get the big picture. Overall, the OS isn't like a process which executes in-order like average user mode code. When the computer boots, the OS will execute in-order to set up its environment. After that, the OS is basically system calls and interrupts.

Each CPU works differently but most CPUs will have an interrupt table mechanism and a syscall mechanism. The interrupt table specifies where to jump for a certain interrupt number and the syscall mechanism is probably one register containing the address of the entry point for a syscall. It works like this on x86-64 (most desktop/laptop computers). x86-64 has the IDT (Interrupt Descriptor Table) and the syscall register is IA32_LSTAR.

The OS isn't written in C because you can call malloc() or anything. The OS is written in C because you can make C static and freestanding (all code needed is in the executable and it doesn't rely on any external software to the executable). Actually, when writing an OS, you cannot call malloc(). You need to avoid any standard library function implementations and use static and freestanding code (the base of C like structs, pointers, arithmetic, variables, etc). C is also used because you can modify arbitrary memory locations with pointers. For example,

unsigned int* ptr = (unsigned int*)0x1234;
*ptr = 0x87654321;

makes sure that address 0x1234 contains 0x87654321. You can also use binary operators (and, or, xor, shift, etc) to modify memory at the bit level.

To answer your question, if you want to define the system calls in an OS that you write yourself, you simply consider your syscalls to be that way. When you write your syscall handler, you consider that someone using your OS knows that a certain syscall number is requesting a certain operation so you oblige by doing that. For example, Linux uses SysV as a convention for syscall numbers and registers used to pass arguments to them (including the syscall number). On x86-64 Linux, from user mode, you put your syscall number in RAX and use the instruction syscall. The processor then looks in IA32_LSTAR for the address of the syscall handler and jumps to it. The processor (the core) is now in kernel mode (in the kernel). The kernel now looks at RAX for the syscall number and answers the request by the doing the associated operation (after several checkups and a bunch of other things).

Related