Title: Knuckles: bringing the database to the data

Authors: Peter Alvaro, Dmitriy V. Ryaboy, Divyakant Agrawal

Addresses: Ask.com, 555 12th Street, Suite 500 Oakland, CA 94607, USA. ' Ask.com, 555 12th Street, Suite 500 Oakland, CA 94607, USA. ' Ask.com, 555 12th Street, Suite 500 Oakland, CA 94607, USA

Abstract: Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. Although click-stream data tends to be collected in a distributed manner to support scaling the servers that host the websites, in general these partitioned click-stream logs are collated and pushed upstream to a centralised database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralised control and is therefore highly scalable. The elimination of centralised control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

Keywords: distributed databases; parallel query processing; peer-to-peer computing; P2P; data partitioning; click-stream data; data warehousing; information management; data retrieval.

DOI: 10.1504/IJCSE.2010.037677

International Journal of Computational Science and Engineering, 2010 Vol.5 No.3/4, pp.214 - 225

Published online: 24 Dec 2010 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article