Title: Evaluation at scale: an approach to evaluate technology for informal workplace learning across contexts
Authors: Stefan Thalmann; Tobias Ley; Ronald Maier; Tamsin Treasure-Jones; Christina Sarigianni; Markus Manhart
Addresses: Graz University of Technology, Institute of Interactive Systems and Data Science and Know-Center GmbH, Graz, Austria ' School of Digital Technologies, Tallinn University, Narva Road 29, 10120 Tallinn, Estonia ' Dept. of Information Systems, Production and Logistics Management, University of Innsbruck, Universitaetsstrasse 15 A-6020 Innsbruck, Austria ' Leeds Institute of Medical Education, University of Leeds, Leeds LS2 9JT, UK ' Dept. of Information Systems, Production and Logistics Management, University of Innsbruck, Universitaetsstrasse 15 A-6020 Innsbruck, Austria ' Apracor GmbH, Munich, Germany
Abstract: The implicit, spontaneous and hidden nature of informal learning in addition to the large and less predictable number of application scenarios challenge the evaluation of learning technology. A further challenge for evaluation is added if a user-centred design method had been employed that already had involved users in large numbers and has led to a high level of expectations regarding the potential of the tool to fulfil their needs. We propose an evaluation approach based on the ideas of active involvement of stakeholders, an iterative planning process, a continuous, open and collaborative interpretation of the data collected on the basis of representations shared across cases plus a pluralism of methods for evaluating learning technology and is particularly intended for large scale evaluations in workplace settings. We present a case, in which we applied the proposed evaluation approach in the context of a large scale research project on technology-enhanced-learning.
Keywords: participatory evaluation; informal learning; workplace learning; participatory design.
International Journal of Technology Enhanced Learning, 2018 Vol.10 No.4, pp.289 - 308
Received: 27 Dec 2016
Accepted: 25 Nov 2017
Published online: 19 May 2018 *