Title: Does the means achieve an end? A document analysis providing an overview of emergency and crisis management evaluation practice in the Netherlands

Authors: Ralf Josef Johanna Beerens

Addresses: Research Department, Institute for Safety (IFV), P.O. Box 7010, 6801 HA, Arnhem, The Netherlands; Division of Risk Management and Societal Safety, Lund University, P.O. Box 118, 221 00 Lund, Sweden

Abstract: Evaluations provide insights into the effectiveness of emergency exercises or the response to a disaster. A well-constructed process is key to capture evidence-based feedback that can support future learning and development. However, little is known about how they are performed in practice and whether they actually meet their intended purpose. Therefore, this paper provides an overview of how 'operational emergency response evaluations' are currently performed in the Netherlands. The study was based on an analysis of evaluation reports and supporting documents from all 25 Dutch safety regions. Outcomes were cross-checked by regional representatives. The findings show that a variety of approaches and designs are currently being used, which are not explicitly or logically linked. Most evaluations are isolated activities that do not build on each other. It is unclear how, and if, lessons identified become lessons learned, while the link between data collection and analysis and conclusions often remains vague. These issues undermine the validity of the evaluation and can have implications for its impact.

Keywords: crisis; disaster; emergency; exercise; evaluation; practice; test; The Netherlands; evaluation design; evaluation process; document analysis; evaluation report.

DOI: 10.1504/IJEM.2019.102310

International Journal of Emergency Management, 2019 Vol.15 No.3, pp.221 - 254

Received: 23 Mar 2018
Accepted: 25 Feb 2019

Published online: 17 Sep 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article