OMPIO: A Parallel I/O Architecture for Open MPI

What is MPI I/O

MPI is the de-facto standard for message passing in parallel scientific applications. MPI-I/O is a part of the MPI-2 specification, and defines file I/O operations for parallel applications. Among the most relevant features of MPI I/O are

What is OMPIO

OMPIO is an implementation of the MPI I/O specification specifically written for Open MPI. The main goals of OMPIO are three fold. First, it increases the modularity of the parallel I/O library by separating functionality into distinct sub-frameworks. Second, it allows frameworks to utilize different run-time decision algorithms to determine which module to use in a particular scenario, enabling non-file system specific decisions. Third, it improves the integration of parallel I/O functions with other components of Open MPI, most notably the derived data type engine and the progress engine. The integration with the derived data type engine allows for faster decoding of derived data types and the usage of optimized data type to data type copy operations.

Overview of the OMPIO Architecture

OMPIO is fundamentally a component of the IO framework in Open MPI. Upon open a file, the OMPIO component initiallizes a number of sub-frameworks and their components, namely The following image summarizes the OMPIO architecture as well as the available components in each framework (date: August 2015).

Where can I download OMPIO?

An early version of OMPIO is included in all releases of Open MPI in the 1.7 and 1.8 series. Since it is however not the default MPI I/O libraries in these releases, users have to explicitely request it through setting an MCA parameter, e.g.
mpirun --mca io ompio -np 64 ./myapplication
Note, that this version of OMPIO is known to have some signficant restrictions. If you want to test the current/stable version, please access the nightly snapshot of Open MPI. OMPIO is scheduled to be the default MPI I/O library in the upcoming 2.x release of Open MPI.

Research Topics in OMPIO

The research focus of the library focuses on exploiting various algorithms for implementing collective I/O operations (e.g. MPI_File_write_all, MPI_File_read_all), automatically determining the optimal number of aggregators and process groups for collective I/O algorithms, developing a selection logic to determine which algorithm to use for optimal performance. Significant efforts have also been invested into the development of scalable and modular shared file pointer algorithms.