I am using the non-blocking MPI_Isend/MPI_Irecv calls in conjunction with repeated MPI_Test calls to achieve communication/computation overlap. I have noticed that whether or not I achieve overlap depends on the frequency at which I call MPI_Test. It seems that if I call it more frequently than a certain threshold frequency, MPI_Test suddenly becomes blocking and uses the thread's resources to progress data. If I call it less frequently than the threshold, then MPI_Test is non-blocking and data progression is masked by the computations. I am not sure what this threshold is, but I've compared the extreme case of calling MPI_Test every iteration against every 500 iterations in my code. The former yields no overlap, and the latter yields complete overlap.
Can someone advise me on whether this is an issue that is commonly encountered and how I can go about computing an optimal MPI_Test frequency for a given function? I imagine that it would be a function of both the size of data transferred and the size of the loop in which my computations occur.