Title: Why the pilot cannot be blamed: a cautionary note about excessive reliance on technology

Authors: Darren Dalcher

Addresses: Software Forensics Centre, Middlesex University, Trent Park, Bramley Road, London N14 4YZ, UK

Abstract: In many human endeavours intelligent automation has taken over some of the tasks traditionally performed by operators, pilots or controllers. The adoption of intelligent protective technology reflects the greater degree of reliability normally ascribed to such systems. Intelligent technology is often credited with saving lives and reducing accidents. This paper looks at the crash of a revolutionary supersonic fighter that resulted from over-reliance on protection technology. The degree of automation of the protection system made it impossible for the pilot to regain control and convince the system that there was a problem. Technology has thus created a new kind of computer-assisted error, where a system designed to make a task safer is actually directly responsible for causing a disaster. Developers thus need to foresee the impact of new technology in its original situational context and consider the implications of wresting control away from the pilot and giving it to the computer.

Keywords: aviation accidents; intelligent technology; ethics; feedback; decision making; responsibility; risk management; safety; software failure; technological disasters; trust; technology reliance; intelligent automation; reliability; supersonic fighters; protection technology; aircraft crashes.

DOI: 10.1504/IJRAM.2007.011988

International Journal of Risk Assessment and Management, 2007 Vol.7 No.3, pp.350 - 366

Published online: 07 Jan 2007 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article