Title: An empirical study of the prediction performance of space-filling designs

Authors: Rachel T. Johnson, Douglas C. Montgomery, Bradley Jones

Addresses: Operations Research Department, Naval Postgraduate School, 1411 Cunningham Road, Monterey, CA 93943, USA. ' School of Computing, Informatics and Decision Systems Engineering, Arizona State University, P.O. Box 878809, Tempe, AZ 85287-8809, USA. ' SAS Institute, SAS Campus Drive, Cary, NC 27513, USA

Abstract: The Gaussian process (GASP) model has found widespread use as a surrogate model for results from deterministic computer model output. In this paper, we compare the fits of GASP models to specific space-filling designs based on their accuracy in predicting responses at previously unsampled locations. This is done empirically using several test functions. We demonstrate that no one space-filling design outperforms another with respect to prediction accuracy. We also found that while the GASP model is substantially easier to fit using the cubic correlation function than with the Gaussian correlation function, its prediction accuracy is not quite as good as the Gaussian correlation function for the chosen test functions especially for larger sample sizes. The best way to improve prediction accuracy is to increase the number of simulation runs, which suggests that the efficient augmentation of space filling designs is an important area for further research.

Keywords: correlation functions; Gaussian process modelling; jackknife plots; sample size; computer experiments; GASP models; space-filling designs; prediction accuracy; simulation.

DOI: 10.1504/IJEDPO.2011.038048

International Journal of Experimental Design and Process Optimisation, 2011 Vol.2 No.1, pp.1 - 18

Published online: 11 Oct 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article