Authors: Afaf G. Bin Saadon; Hoda M.O. Mokhtar
Addresses: Faculty of Computers and Information, Cairo University, Giza, 12613, Egypt ' Faculty of Computers and Information, Cairo University, Giza, 12613, Egypt
Abstract: Iterative computation has become increasingly needed for a large and important class of applications such as machine learning and data mining. These iterative applications typically apply computations over large-scale datasets. So it is desirable to develop efficiently distributed frameworks to process data iteratively. On the other hand, data keeps growing over time as new entries are added and existing entries are deleted or modified. This incremental nature of data makes the previously computed results of iterative applications stale and inaccurate over time. It is hence necessary to periodically refresh the computation so that the new changes can be quickly reflected in the computed results. This paper presents the existing distributed systems that support iterative and incremental computations on large-scale datasets. It describes the main optimisations and features of these systems and identifies their limitations.
Keywords: big data; distributed systems; iterative computation; incremental processing.
International Journal of Data Science, 2019 Vol.4 No.1, pp.18 - 30
Received: 02 May 2017
Accepted: 25 Aug 2017
Published online: 11 Mar 2019 *