Title: SPEC HPG benchmarks for high-performance systems

Authors: Matthias S. Muller, Kumaran Kalyanasundaram, Greg Gaertner, Wesley Jones, Rudolf Eigenmann, Ron Lieberman, Matthijs Van Waveren, Brian Whitney

Addresses: High Performance Computing Center Stuttgart (HLRS), Noblestr.19, 70569 Stuttgart, Germany. ' SPEC HPG Chair, Silicon Graphics, Inc.,1500 Crittenden Lane, Mountain View, CA 94043, USA. ' 3 NewcastleWay Bedford, NH 03110, USA. ' National Renewable Energy Laboratory, 1617 Cole Blvd., Golden, CO 80401-3393, USA. ' Purdue University, School of Electrical and Computer Engineering, Electrical Engineering Building, 465 Northwestern Ave., West Lafayette, IN 47907-2035, USA. ' Hewlett-Packard Company, 3000 Waterview Parkway, Richardson, TX 75080, USA. ' SPEC HPG Secretary, Fujitsu Systems Europe Ltd., 8, rue Maryse Hilsz, Parc de la Plaine, 31500 Toulouse, France. ' Sun Microsystems, Inc., 3295 NW 211th Terrace, Hillsboro, OR 97124, USA

Abstract: In this paper, we discuss the results and characteristics of the benchmark suites maintained by the Standard Performance Evaluation Corporation|s (SPEC) High-Performance Group (HPG). Currently, SPECHPGhas two lines of benchmark suites for measuring performance of large-scale systems SPEC OMP and SPEC HPC2002. SPEC OMP uses the OpenMP API and includes benchmark suites intended for measuring performance of modern shared memory parallel systems. SPEC HPC2002 uses both OpenMP and MPI, and thus it is suitable for distributed memory systems, shared memory systems and hybrid systems. SPEC HPC2002 contains benchmarks from three popular application areas: chemistry, seismic and weather forecasting. Each of the three benchmarks in HPC2002 has a small and a medium data set in order to satisfy the need for benchmarking a wide range of high-performance systems. We analyse published results of these benchmark suites regarding scalability. We also present current efforts of SPEC HPG to create new releases of the benchmark suites.

Keywords: SPEC OMP; SPEC HPC; benchmarking; high-performance computing; performance evaluation; openMP; MPI; large-scale systems; performance measurement; shared memory parallel systems; distributed memory systems; chemistry; seismic modelling; weather forecasting; earthquake modelling; environmental pollution; crash simulation.

DOI: 10.1504/IJHPCN.2004.008345

International Journal of High Performance Computing and Networking, 2004 Vol.1 No.4, pp.162 - 170

Available online: 07 Dec 2005 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article