Title: Linear vs. quadratic discriminant analysis classifier: a tutorial

Authors: Alaa Tharwat

Addresses: Electrical Department, Faculty of Engineering, Suez Canal University, Ismailia, Egypt

Abstract: The aim of this paper is to collect in one place the basic background needed to understand the discriminant analysis (DA) classifier to make the reader of all levels be able to get a better understanding of the DA and to know how to apply this classifier in different applications. This paper starts with basic mathematical definitions of the DA steps with visual explanations of these steps. Moreover, in a step-by-step approach, a number of numerical examples were illustrated to show how to calculate the discriminant functions and decision boundaries when the covariance matrices of all classes were common or not. The singularity problem of DA was explained and some of the state-of-the-art solutions to this problem were highlighted with numerical illustrations. An experiment is conducted to compare between the linear and quadratic classifiers and to show how to solve the singularity problem when high-dimensional datasets are used.

Keywords: linear discriminant classifier; LDC; quadratic discriminant classifier; QDC; classification; singularity problem; discriminant function; decision boundaries; subspace method; regularised LDA; linear discriminant analysis; RLDA; quadratic discriminant analysis; QDA.

DOI: 10.1504/IJAPR.2016.079050

International Journal of Applied Pattern Recognition, 2016 Vol.3 No.2, pp.145 - 180

Received: 14 Mar 2016
Accepted: 29 Mar 2016
Published online: 10 Sep 2016 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article