Will processes be blocked, if the number of child processes greater than processors while using pipeline?

Viewed 70

My program stops running while the number of child processes is big. I do not know what the problem could be, but I guess the child processes are somehow blocked while running.

Here is the main workflow of the program:

void function(int process_num){

    int i;

    // initial variables for fork()
    int status = 0;
    pid_t child_pid[process_num], wpid;
    int *fds = malloc(sizeof(int) * process_num * 2);

    // initial pipes 
    for(i=0; i<process_num; i++){
        if(pipe(fds + i*2) <0)
            exit(0);
    }

    // start child processes to write
    for(i=0; i<process_num; i++){
        child_pid[i] =fork();

        if(child_pid[i] == 0){
            close(fds[i*2]);
            // do something ...
            // write(fds[i*2+1], something);
            close(fds[i*2+1]);
            exit(0);
        }else if(child_pid[i] == -1){
            printf("fork error\n");
            exit(0);
        }
    }

    // parent waits child processes and reads
    for(i=0; i<process_num; i++){

      pid_t cpid = waitpid(child_pid[i], &status, 0);
      if (WIFEXITED(status)){
        close(fds[i*2+1]);
        // do something ...
        // read(fds[i*2], something);
       close(fds[i*2]);
      }
    }
    free(fds);
    while((wpid = wait(&status)) > 0);
}

I checked the status of processes via htop, there were several (e.g. 8 while process_num was 110) child processes left with state S.

and now comes my question: if the number of child processes is greater than the number of processors, will the child processes be blocked while using pipeline for child processes and present process to communicate (parent process waits until all child processes executed)? Thanks a lot!

EDIT: I printed the Id of fds when using read() and write(), and I found that the read began at 4 and write at 5, I have no idea why was it the case, does somebody know that?

1 Answers

The number of processors has no impact here. The operating system is alive, able to run any process that has something to do. This is a pure software problem, all processes are in sleep state (S) waiting for some event that never happens.

Related