Title: Gain parameter and dropout-based fine tuning of deep networks

Authors: M. Arif Wani; Saduf Afzal

Addresses: Department of Computer Science, University of Kashmir, India ' Department of Computer Science, University of Kashmir, India

Abstract: Training of deep neural networks can involve two phases: unsupervised pre-training and supervised fine tuning. Unsupervised pre-training is used to learn the initial parameter values of deep networks, while as supervised fine tuning improves upon what has been learned in the pre-training stage. Backpropagation algorithm can be used for supervised fine tuning of deep neural networks. In this paper we evaluate the use of backpropagation with gain parameter algorithm for fine tuning of deep networks. We further propose a modification where backpropagation with gain parameter algorithm is integrated with the dropout technique and evaluate its performance in fine tuning of deep networks. The effectiveness of fine tuning done by proposed technique is also compared with other variants of backpropagation algorithm on benchmark datasets. The experimental results show that the fine tuning of deep networks using the proposed technique yields promising results among all the studied methods on the tested datasets.

Keywords: deep learning; deep neural networks; fine tuning; gain parameter; dropout technique.

DOI: 10.1504/IJIIDS.2018.096584

International Journal of Intelligent Information and Database Systems, 2018 Vol.11 No.4, pp.236 - 254

Received: 05 Jul 2017
Accepted: 20 Dec 2017
Published online: 06 Dec 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article