I have been exploring parallel IO in Chapel. The Chapel documentation mentions a parallel IO flag, and that channels can work in parallel. But, I don't see anything else.
I don't have a particular question in mind, but just want to know more about it.
Could Chapel team or sound Chapel practitioners discuss any example of a due use of the Chapel parallel IO paradigm?
Parallel I/O can mean different things to different people, which makes it challenging to have a single, simple answer to this (though it might suggest that the Chapel project should add a parallel I/O landing page to its documentation which would point to other resources?). For example, "parallel I/O" could mean:
Another important factor is the desired file format: text, binary, or a specific file format like HDF5, NetCDF, etc.
Generally speaking, the explicit way of doing parallel I/O in Chapel is to create a number of tasks using Chapel's language features for expressing parallelism (e.g., coforall, cobegin, or begin), and then to give each task its own channel to read from / write to. If all of the channels refer to a single file, the tasks would likely need to coordinate between themselves to make sure they were writing to / reading from disjoint segments of the file. If each channel refers to its own file, such coordination wouldn't be necessary.
The other major way to get parallel I/O in Chapel is implicit, by invoking a library routine where the parallelism is created and managed within the routine itself—either using techniques like the above for routines written in Chapel, or by calling out to an external parallel function (e.g., a parallel I/O routine from a C library).
Finally, you could create multiple tasks that call serial (or parallel) I/O library routines simultaneously.
For an example of the first, explicit approach, see this sample program that I recently put together in response to a similar question. It declares a 2D array whose rows are block-distributed and then uses a task per locale (compute node) to write that locale's sub-array out to a single/shared binary-format file. It then does a similar thing to read the data back into a second array and verifies that the two arrays match. In both cases, each task advances its channel to the appropriate file offset corresponding to the values it wants to write/read.
Examples of the library-based approach to parallel I/O include the hdf5WriteDistributedArray() routine which logically does something very similar to the previous example, yet using the HDF5 file format. Or, the readAllHDF5Files() routine is an example of a library routine that reads from multiple files in parallel.
I think it's safe to say that Chapel should support many more library routines to help with parallel I/O than it has today. The main challenge is knowing which patterns and formats from the space outlined above will be most important to users. We're always open to requests and input in this regard.
