UHIO: optimizing collective I/O operations for Open MPI
MPI is the de-facto standard for message passing in parallel scientific applications. MPI-I/O is a part of the MPI-2 specification, and defines file I/O operations for parallel applications. Among the most relevant features of MPI I/O are
- blocking and non-blocking I/O operations
- individual and collective I/O operations
- relaxed consistency semantics
- pre-definition of access patterns
- ability to handle application specific hints
UHIO is a partial implementation of the MPI I/O specification. The
library focuses on exploiting various algorithms for implementing
collective I/O operations (e.g. MPI_File_write_all,
MPI_File_read_all). The current version of the library is implemented
using the profiling interface of MPI.
As of today, four different classes of algorithms have been implemented and evaluated:
- dynamic segmentation algorithms: re-arrange user data of multiple processes
in order to optimize disk access
- static segmentation algorithms: re-arrange user data of multiple processes
optimizing the communication between the processes
- individual algorithms: each process handles its own data items
- individual scheduled algorithms: similar to the individual algorithm, with
additional scheduling of the processes to prevent congestion on the
I/O level.
