What is the proper way to benchmark Julia code that uses MPI?

user_14709952 0
Viewed 147

I am learning to write Julia code using MPI, and I would like to benchmark this basic send/receive example. I made a slight modification to the example code to include timing.

using MPI

function executeSendReceive(n)
    comm = MPI.COMM_WORLD
    rank = MPI.Comm_rank(comm)
    size = MPI.Comm_size(comm)

    dst = mod(rank+1, size)
    src = mod(rank-1, size)

    N = 2^n

    send_mesg = Array{Int8}(undef, N)
    recv_mesg = Array{Int8}(undef, N)

    fill!(send_mesg, Int8(rank))

    rreq = MPI.Irecv!(recv_mesg, src,  src+32, comm)

    print("$rank: Sending   $rank -> $dst = $send_mesg\n")
    sreq = MPI.Isend(send_mesg, dst, rank+32, comm)

    stats = MPI.Waitall!([rreq, sreq])

    print("$rank: Received $src -> $rank = $recv_mesg\n")

    MPI.Barrier(comm)
end

MPI.Init()
@time executeSendReceive(2)
MPI.Finalize()

However, when I run the script in the terminal using 2 processes, I get some messy-looking output.

0: Sending   0 -> 1 = Int8[0, 0, 0, 0]
1: Sending   1 -> 0 = Int8[1, 1, 1, 1]
0: Received 1 -> 0 = Int8[1, 1, 1, 1]
1: Received 0 -> 1 = Int8[0, 0, 0, 0]
    0.077899 seconds0.077898 seconds (3.37 k allocations: 187.234 KiB) (3.37 k allocations: 187.234 KiB)

How can fix this formatting issue? I would also like to repeat the procedure for several message sizes, but I don't know how to manipulate the output given by @time. Can somebody point me in the right direction?

julia mpi benchmarking
1 Answers

My first suggestion is to take a look here: https://www.geeksforgeeks.org/benchmarking-in-julia/ at some more benchmarking routines. Using @time is generally not a reliable way to benchmark unless you are trying to get a very high-level understanding of the performance.

It also looks like single-line formatting from your output is the result of the way you are structuring and subsequently running the code. Try running it without the two processes or adding a println() after the @time.

Either way, what you may be looking for is: https://docs.julialang.org/en/v1/base/base/#Base.@timed which allows you to work with and print out specific parts of the timing info that may be relevant to you. I suggest looking around the docs in that link above as it has all of the relevant functions there.

user_76198080
Related