Title: Joint variable and variable projection algorithms for separable nonlinear models using Aitken acceleration technique
Authors: Lianyuan Cheng; Jing Chen; Yingjiao Rong
Addresses: School of Science, Jiangnan University, Wuxi, 214122, China ' The Science and Technology on Near-Surface Detection Laboratory, Wuxi, China; School of Science, Jiangnan University, Wuxi, 214122, China ' The Science and Technology on Near-Surface Detection Laboratory, Wuxi, China
Abstract: This paper proposes a joint variable-based gradient descent algorithm (Joint-GD) and a variable projection (VP)-based gradient descent algorithm (VP-GD) for separable nonlinear models. The VP algorithm takes advantage of the separability property of variables to reduce the dimensionality of the parameters, which makes the convergence rates faster. In order to speed up the convergence of the gradient descent algorithm, the Aitken acceleration technique is introduced in the algorithms, which is second-order convergent. Moreover, the Aitken-based methods are robust to the step-size, therefore they can be widely used in engineering practices. The numerical simulation shows the effectiveness of the proposed algorithms.
Keywords: variable projection algorithm; joint variable algorithm; gradient descent algorithm; separable nonlinear model; Aitken acceleration technique.
International Journal of Modelling, Identification and Control, 2023 Vol.42 No.3, pp.202 - 210
Received: 12 Jan 2022
Accepted: 04 Apr 2022
Published online: 05 Apr 2023 *