Title: Scaling up genome annotation using MAKER and work queue

Authors: Andrew Thrasher; Zachary Musgrave; Brian Kachmarck; Douglas Thain; Scott Emrich

Addresses: Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA ' Yelp, Inc., San Francisco, CA, USA ' Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA ' Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA ' Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA

Abstract: Next generation sequencing technologies have enabled sequencing many genomes. Because of the overall increasing demand and the inherent parallelism available in many required analyses, these bioinformatics applications should ideally run on clusters, clouds and/or grids. We present a modified annotation framework that achieves a speed-up of 45x using 50 workers using a Caenorhabditis japonica test case. We also evaluate these modifications within the Amazon EC2 cloud framework. The underlying genome annotation (MAKER) is parallelised as an MPI application. Our framework enables it to now run without MPI while utilising a wide variety of distributed computing resources. This parallel framework also allows easy explicit data transfer, which helps overcome a major limitation of bioinformatics tools that often rely on shared file systems. Combined, our proposed framework can be used, even during early stages of development, to easily run sequence analysis tools on clusters, grids and clouds.

Keywords: genome annotation; work queue; distributed computing; next generation sequencing; bioinformatics; Caenorhabditis japonica; explicit data transfer; clusters; grid computing; cloud computing.

DOI: 10.1504/IJBRA.2014.062994

International Journal of Bioinformatics Research and Applications, 2014 Vol.10 No.4/5, pp.447 - 460

Published online: 24 Oct 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article