"Primary job terminated normally but 1 process returned a non-zero exit code" only in some cases

Viewed 4259

Good afternoon, I've developed a 2D FFT in MPI for scientific purpose. Everything used to work until I've implemented MPI_Scatterv. Since I've implemented it something odd started happening. In particular if I stay below 64 modes I don't get problems, but when I push above it I get the message:

> Primary job  terminated normally, but 1 process returned
> a non-zero exit code. Per user-direction, the job has been aborted.
>-------------------------------------------------------------------------- 

>mpiexec noticed that process rank 0 with PID 0 on node MacBook-Pro-di-Mirco
>exited on signal 11 (Segmentation fault: 11).`

I can't figure out where is the mistake, but I'm pretty sure it is in MPI_Scatterv. Could anyone help me please?

/********************************** Setup factors for scattering **********************************/
  // Alloc the arrays
  int* displs = (int *)malloc(size*sizeof(int));
  int* scounts = (int *)malloc(size*sizeof(int));
  int* receive = (int *)malloc(size*sizeof(int));

  // Setup matrix
  int modes_per_proc[size];
  for (int i = 0; i < size; i++){
  modes_per_proc[i] = 0;
  }


  // Set modes per processor
  cores_handler( nx*nz, size, modes_per_proc);

  // Scattering parameters
  for (int i=0; i<size; ++i) {
  scounts[i] = modes_per_proc[i]*ny*2;
  receive[i] = scounts[i];
  displs[i] = displs[i-1] + modes_per_proc[i-1] *ny*2;  // *2 to handle complex numbers
  if (i == 0 ) displs[0] = 0;
   }


   /************************************************ Data scattering ***********************************************/
  MPI_Scatterv(U, scounts, displs, MPI_DOUBLE, u, receive[rank] , MPI_DOUBLE, 0, MPI_COMM_WORLD);
  MPI_Barrier(MPI_COMM_WORLD);

The core_handler function:

void cores_handler( int modes, int size, int modes_per_proc[size]) {
int rank =0;
int check=0;

for (int i = 0; i < modes; i++) {
    modes_per_proc[rank] = modes_per_proc[rank]+1;
    rank = rank+1;
    if (rank == size ) rank = 0;
}

for (int i = 0; i < size; i++){
//printf("%d modes on rank %d\n", modes_per_proc[i], i);
check = check+modes_per_proc[i];
}
if ( (int)(check - modes) != 0 ) {
        printf("[ERROR] check - modes = %d!!\nUnable to scatter modes properly\nAbort... \n", check - modes);
}
0 Answers
Related