First SO question, so here it goes.
I'm not asking for someone to review the code, i want to get to the bottom of this. It would be helpful if someone knew what change in the kernel could be responsible for the following.
In the University we were tasked to implement extended functionality in a modeled Operating System written in C (written by my professor), that models each core with a pthread.
Project github forked by me.
We had to implement the necessary functionality by implementing the required syscalls. (multithreading, sockets, pipes, mlfq, etc).
After implementing each functionality we had to confirm that it was working using the validate_api program.
Problem time:
The validate_api.c contains a lot of tests to check the functionality of the OS.
BOOT_TEST: bare-boots the machine and tests something.
A simple test for creating a new thread inside a process:
BOOT_TEST(test_create_join_thread,
"Test that a process thread can be created and joined. Also, that "
"the argument of the thread is passed correctly."
)
{
int flag = 0;
int task(int argl, void* args) {
ASSERT(args == &flag);
*(int*)args = 1;
return 2;
}
Tid_t t = CreateThread(task, sizeof(flag), &flag);
/* Success in creating thread */
ASSERT(t!=NOTHREAD);
int exitval;
/* Join should succeed */
ASSERT(ThreadJoin(t, &exitval)==0);
/* Exit status should be correct */
ASSERT(exitval==2);
/* Shared variable should be updates */
ASSERT(flag==1);
/* A second Join should fail! */
ASSERT(ThreadJoin(t, NULL)==-1);
return 0;
}
As you can see there is a nested function called task() which is the starting point of the thread that is going to be created using the createThread() syscall we implemented.
The problem is that, although the thread is created correctly, when scheduled to run, the program exits with segmentation fault and cannot access the memory of the task function, gdb doesn't even recognize it as a variable (in the thread struct field pointing to it). The weird thing is that this happens ONLY when using a kernel version newer than 5.7. I opened an issue in the original project's repo.
Running the actual OS and its programs it's fine with no issues whatsoever, only validate_api fails due to that nested function. If i move the task function into global scope then the test finishes successfully. Same goes for every other test that has a nested function inside.
Note: The project is finished (1 month now), i downgraded to 5.4 just to test my implementation.
Note2: I dont need help with the implementation of any functionality (the project is finished any way), i just want to figure out why it doesn't work on kernels > 5.7
Note3: I'm here because my prof. doesn't respond to my repeated emails regarding the issue
I tried compiling using -fno-stack-protector and with -z execstack with no luck. Also simple nested functions like:
int main(){
int foo(){
puts("Hello there");
}
foo();
}
work with any kernel
Machine Details:
Arch Linux - 5.10 / 5.4 LTS
GCC 10.2
Thank you
UPDATE: The test joins the thread, so it never goes out of scope.