Title: On trustworthy measurements when testing dependable systems: a discussion and experiences

Authors: Andrea Ceccarelli; Andrea Bondavalli

Addresses: Department of Mathematics and Informatics, University of Florence, Viale Morgagni 65, Firenze, Italy ' Department of Mathematics and Informatics, University of Florence, Viale Morgagni 65, Firenze, Italy

Abstract: The scientific literature and the industrial practice agree since many years on the fundamental role of experimental evaluation (testing) of critical systems for the assessment of the dependability attributes, and consequently on the relevance of achieving trustworthy measurements. This paper discusses and motivates with the support of three case studies, the possible role of the body of knowledge offered by measurement theory (metrology) to quantitatively assess the quality of measuring instruments, i.e., the tools, and the results collected. The paper first introduces notions of metrology and contextualises them for the dependability evaluation of systems. Successively it presents three case studies developed by the authors where attention to principles from measurement theory and the metrological assessment of tools and results are carried out. The paper ultimately reviews the main guidelines identified discussing their application in the case studies.

Keywords: experimental evaluation; critical systems; metrology; measurement theory; dependability evaluation; localisation; clock; system testing; fault injection; uncertainty; resolution; intrusiveness; repeatability; trustworthy measurements; dependable systems; measuring instruments; metrological assessment.

DOI: 10.1504/IJCCBS.2015.073543

International Journal of Critical Computer-Based Systems, 2015 Vol.6 No.2, pp.154 - 170

Received: 25 Feb 2015
Accepted: 23 Sep 2015

Published online: 11 Dec 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article