Title: Analysis on fast training speed of extreme learning machine and replacement policy

Authors: Shi-Xin Zhao; Xi-Zhao Wang; Li-Ying Wang; Jun-Mei Hu; Wei-Ping Li

Addresses: College of Management, Hebei University, Baoding 071002, China; Department of Mathematics and Physics, Shijiazhuang Tiedao University, Shijiazhuang 050043, China ' College of Computer Science and Soft Engineering, Shenzhen University, Shenzhen 518060, China ' Department of Mathematics and Physics, Shijiazhuang Tiedao University, Shijiazhuang 050043, China ' Department of Mathematics and Physics, Shijiazhuang Tiedao University, Shijiazhuang 050043, China ' School of Economics and Management, Beijing Jiaotong University, Beijing 100044, China

Abstract: Extreme learning machine is known for its fast learning speed while maintaining acceptable generalisation. Its learning process can be divided into two parts: (1) randomly assigns input weights and biases in hidden layer, and (2) analytically determines output weights by the use of Moore-Penrose generalised inverse. Through the analysis from theory and experiment aspects we point out that it is the random weights assignment rather than the analytical determination with generalised inverse that leads to its fast training speed. In fact, the calculation of generalised inverse of hidden layer output matrix based on singular value decomposition (SVD) has very low efficiency especially on large scale data, and even directly cannot work. Considering this high calculation complexity reduces the learning speed of ELM conjugate gradient is introduced as a replacement of Moore-Penrose generalised inverse and conjugate gradient based ELM (CG-ELM) is proposed. Numerical simulations show that, in most cases, CG-ELM achieved faster speed than ELM in the condition of maintaining similar generalisation. Even in the case that ELM cannot work because of the huge amount of data CG-ELM attains good performance, which illustrates that Moore-Penrose generalised inverse is not the contribution of fast learning speed of ELM from experiment view.

Keywords: extreme learning machine; generalised inverse; SVD; conjugate gradient method.

DOI: 10.1504/IJWMC.2017.089327

International Journal of Wireless and Mobile Computing, 2017 Vol.13 No.4, pp.314 - 322

Received: 14 Jun 2017
Accepted: 18 Aug 2017

Published online: 11 Jan 2018 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article