Scalable data management for map-reduce-based data-intensive applications: a view for cloud and hybrid infrastructures
by Gabriel Antoniu; Alexandru Costan; Julien Bigot; Frédéric Desprez; Gilles Fedak; Sylvain Gault; Christian Pérez; Anthony Simonet; Bing Tang; Christophe Blanchet; Raphael Terreux; Luc Bougé; François Briant; Franck Cappello; Kate Keahey; Bogdan Nicolae; Frédéric Suter
International Journal of Cloud Computing (IJCC), Vol. 2, No. 2/3, 2013

Abstract: As map-reduce emerges as a leading programming paradigm for data-intensive computing, today's frameworks which support it still have substantial shortcomings that limit its potential scalability. In this paper, we discuss several directions where there is room for such progress: they concern storage efficiency under massive data access concurrency, scheduling, volatility and fault-tolerance. We place our discussion in the perspective of the current evolution towards an increasing integration of large-scale distributed platforms (clouds, cloud federations, enterprise desktop grids, etc.). We propose an approach which aims to overcome the current limitations of existing map-reduce frameworks, in order to achieve scalable, concurrency-optimised, fault-tolerant map-reduce data processing on hybrid infrastructures. This approach will be evaluated with real-life bio-informatics applications on existing Nimbus-powered cloud testbeds interconnected with desktop grids.

Online publication date: Fri, 28-Feb-2014

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Cloud Computing (IJCC):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com