How can an operating system detect an out of range segmentation fault in C?

user_16944784 0
Viewed 168

I encounter this problem when learning Operating System and I'm really interested in how operating system detecs whether an array index is out of range and therefore produce a segmentation fault?

int main(){
    char* ptr0;
    ptr0[0] = 1;
}

The code above will absolutely produce a segmentation fault, since ptr0 is not allocated with any memory.

But if add one line, things change.

int main(){
    char* ptr0;
    char* ptr1 = ptr0;
    ptr0[0] = 1;
}

This code won't cause any fault, even you change ptr0[0] to ptr0[1000], it still won't cause any segmentation fault.

I don't know why the line has such power

char* ptr1 = ptr0

I tried to disassmbly these codes but find little information.

Could somebody explain that to me on the perspective of memory allocation? thanks a lot.

c c++
2 Answers

A segmentation fault happens when a process attempts to access memory it's not supposed to, not necessarily if an array is read out of bounds.

In your particular case the variable ptr0 is uninitialized, and so if you attempt to read it any value may be read and it need not even be consistent. So in the case of the first program the value that was read happened to be an invalid memory address and attempting to read from that address triggered a sigfault, while in the case of the second program the value read happened to be a valid address for the program and so a segfault was not generated.

When I ran these programs, both resulted in a segfault. This demonstrates the undefined behavior present in the program which attempts to dereference an invalid pointer.

When a program has undefined behavior, the C standard makes no guarantees regarding what the program will do. It may crash, it may output unexpected results, or it may appear to work properly.

user_16871190

The operating system and the hardware maintain a map of memory in the process’ virtual address space. Each time the process accesses memory, the hardware consults the map information and decides whether or not the access is allowed. If the access is not allowed, it generates a trap to alert the operating system.

This process will catch many incorrect accesses—it will catch all those that attempt to read or write memory that is not mapped or that attempt to write memory that is mapped read-only. It will not catch all incorrect accesses—it will not catch accesses that are to a wrong location (in terms of what the program’s author or user desires) but that are within mapped memory and the appropriate permissions.

Commonly, the operating system’s map information is more complete than the hardware’s. The operating system may map only a subset of the a process’ address space. This is because processes often do not use all of their address space (code and data to handle rare errors is often not executed or used) or do not use all of it all the time (processes spend some time doing one task before going on to another, and perhaps later returning to an earlier task). So, when the hardware reports a memory access fault to the operating system, the operating system will consult its complete information. If the process is allowed to access the attempted location, the operating system will set up physical memory as necessary, update the map for the hardware, and resume execution of the process. If the process is not allowed to access the attempted location, the operating system will report a signal to the process or terminate it or take other appropriate action.

The code above [char *ptr0; ptr0[0] = 1;] will absolutely produce a segmentation fault, since ptr0 is not allocated with any memory.

This is false, for several reasons:

  • Since ptr0 is not initialized, its value is indeterminate. When calculating the address for ptr0[0], the compiler will not necessarily use an address outside the process’ address space; it might use an address that is inside the address space and writable. In this case, storing 1 in that location will not generate a segmentation fault.
  • Due to a special rule in the C standard (C 2018 6.3.2.1 2), using the uninitialized object ptr0 in this situation results in the behavior of the program not being defined by the C standard. The compiler may transform this compiler to any other program.
  • Even if ptr0 were defined, the compiler may observe that the program has no defined observable behavior—it does not read any input, print anything, write anything to files, or change any volatile objects. So the compiler may optimize the program by changing it to an empty main function that does nothing.

But if add one line, things change.

If there is any change here, using char *ptr0; char *ptr1 = ptr0;, it is mere happenstance of the compiler. This program has the same semantics as the earlier program.

From your other comments, you might have intended to write char ptr0[0]; instead of char *ptr;. Zero-length arrays are not defined by the C standard. However, some compilers may allow them, as an extension to the C standard. In this case, what likely happens is that the compiler picks a location to put the array, likely on the stack. Then ptr0[0] = 1; attempts to store a byte at that location. Although the array has been assigned a location, zero bytes there are reserved for it. Instead, those bytes may be in use for something else. Possibly they are the function return address or possibly they are just filler used to help align the stack. In this case, ptr0[0] = 1; might overwrite necessary data for your program and break it. Or it might overwrite unused data and have no effect. Or, again, the behavior of your program is not defined by the C standard, so the compiler might transform it in other ways.

user_2982250
Related