Title: A dataflow runtime environment and static scheduler for edge, fog and in-situ computing

Authors: Caio B.G. Carvalho; Victor C. Ferreira; Felipe M.G. França; Cristiana B. Bentes; Gabriele Mencagli; Tiago A.O. Alves; Alexandre C. Sena; Leandro A.J. Marzulo

Addresses: Programa de Engenharia de Sistemas e Computação - COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ, Brazil ' Programa de Engenharia de Sistemas e Computação - COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ, Brazil ' Programa de Engenharia de Sistemas e Computação - COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ, Brazil ' Departamento de Engenharia de Sistemas e Computação, Faculdade de Engenharia, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil ' Department of Computer Science, University of Pisa, Pisa, Italy ' Departamento de Informática e Ciência da Computação, Instituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil ' Departamento de Informática e Ciência da Computação, Instituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil ' Departamento de Informática e Ciência da Computação, Instituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil

Abstract: In the dataflow computation model, tasks are executed according to data dependencies, instead of following program order, enabling natural parallelism exploitation. Sucuri is a dataflow library for Python that allows transparent execution of applications on clusters of multicores, while taking care of scheduling issues. Recent trends in edge/fog/In-situ computing assume that storage and network devices will have processing elements with lower power consumption and performance, which would make a good case for runtime environments that deal with the data versus computation movements trade-off in a more transparent and automated way. This work presents a study on different factors that should be considered when running dataflow applications in in-situ environments, using Sucuri to conduct experiments in a small system emulating a smart storage (in-situ device) utilisation. A static scheduling solution is also presented, allowing Sucuri to choose the most suited approach regarding this in-situ trade-off.

Keywords: dataflow computing; edge computing; fog computing; scheduling techniques; smart storage.

DOI: 10.1504/IJGUC.2019.099685

International Journal of Grid and Utility Computing, 2019 Vol.10 No.3, pp.235 - 247

Received: 06 Mar 2018
Accepted: 20 Jul 2018

Published online: 20 May 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article