Title: The impact of applications' I/O strategies on the performance of the Lustre parallel file system

Authors: Francieli Zanon Boito, Rodrigo Virote Kassick, Philippe O.A. Navaux

Addresses: Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil. ' Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil. ' Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil

Abstract: Parallel applications present multiple approaches regarding the management of data. Due to specific characteristics of parallel file systems, some approaches will provide better performance than others due to a better matching to the system|s internals. One common situation is when each instance of an application accesses exclusive data stored in the file system. This paper studies some I/O techniques for this situation and evaluates them on the Lustre file system. We provide a guide to help developers tune their application to extract the best performance out of Lustre. Our results show expressive gains in performance related with the choice of access pattern of the application. We present considerations on operation granularity, intra-node concurrency and temporal behaviour of the application.

Keywords: parallel file systems; PFS; Lustre; parallel I/O; access patterns; HPC; clustering; I/O granularity; scalability; I/O strategies; data organisation; input output; intra-node concurrency; temporal behaviour; high performance systems.

DOI: 10.1504/IJHPSA.2011.040465

International Journal of High Performance Systems Architecture, 2011 Vol.3 No.2/3, pp.122 - 136

Published online: 30 May 2011 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article