Is there a way to Send (capital) list of string by using Mpi4py

user_14512388 0
Viewed 250

Here is my dataset in rank 0, and I want to send it to another rank by using Send after converting it to NumPy array.

data = [['zzz', '2', '-1'], ['xxx', '5', '-99'], ['zzz', '2', '-1']]

I tried many string datatype when converting to a NumPy array, but it doesn't work.

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

if rank == 0:
    data = [['zzz', '2', '-1'], ['xxx', '5', '-99'], ['zzz', '2', '-1']]
    data_arr = np.array(data, dtype='S13')
    comm.Send(data, dest=1, tag=10)
else:
    data_arr = np.empty(shape=(3,3),dtype='S13')
    comm.Recv(data_arr, source=0, tag=10)

I know send and recv (lower) can handle this, but I am still wondering it is possible to passing array or list of string by using Send and Recv.

Many thanks!

mpi4py numpy-ndarray numpy python
2 Answers

You can transform your numpy array to bytes using, send it over and then transform it back to a numpy array.

Some nuances are that you need to know the shape of your array and the dtype in order to transform it successfully. Based on your example a solution could be:

import numpy as np

from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

if rank == 0:
    data = [['zzz', '2', '-1'], ['xxx', '5', '-99'], ['zzz', '2', '-1']]
    data_arr = np.array(data, dtype='S13')
    # Transform the data to bytes and send it
    comm.Send(data_arr.tobytes(),dest=1)
else:
    # Initialize a bytearray big enough to fit our message
    data_arr = bytearray(500)
    comm.Recv(data_arr, source=0)
    data_arr = np.frombuffer(data_arr, dtype='S13', count=9)
    data_arr = data_arr.reshape((3, 3))
    print(data_arr)
user_81214360

I asked the single string version of this question and got a great answer: mpi4py Send characters

Perhaps you could build a larger string with delimiter characters, send it as a single string and split it later? Not ideal but functional.

user_83432280
Related