Title: Layered evaluation of adaptive learning systems

Authors: Peter Brusilovsky, Charalampos Karagiannidis, Demetrios Sampson

Addresses: School of Information Sciences, University of Pittsburgh, USA. ' Informatics and Telematics Institute, Centre for Research and Technology Hellas, and Department of Cultural Technology and Communication, Aegean University, Mytilini, Greece. ' Informatics and Telematics Institute, Centre for Research and Technology Hellas, and Department of Technology Education and Digital Systems, University of Piraeus, 150, Androutsou Street, Piraeus, GR 18534, Greece

Abstract: This paper suggests an alternative to the traditional ||as a whole|| approach of evaluating adaptive learning systems (ALS), and adaptive systems, in general. We argue that the commonly recognised models of adaptive systems can be used as a basis for a layered evaluation that offers certain benefits to the developers of ALS. Therefore, we propose the layered evaluation framework, where the success of adaptation is addressed at two distinct layers: user modelling and adaptation decision making. We outline how layered evaluation can improve the current evaluation practice of ALS. To build a stronger case for a layered evaluation, we re-visit the evaluation of the InterBook where the layered approach can provide a difference and provide an example of its use in KOD learning system.

Keywords: adaptive learning systems; adaptive hypermedia; user modelling; layered evaluation; knowledge-on-demand; web-based learning; internet.

DOI: 10.1504/IJCEELL.2004.005729

International Journal of Continuing Engineering Education and Life-Long Learning, 2004 Vol.14 No.4/5, pp.402 - 421

Published online: 28 Nov 2004 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article