Article: Parallel hierarchical clustering using weighted confidence affinity Journal: International Journal of Data Mining, Modelling and Management (IJDMMM) 2011 Vol.3 No.2 pp.110 - 129 Abstract: There have been many attempts for clustering categorical data such as market basket dataset. However, most of categorical clustering approaches belong to partitional clustering which requires at least one input parameter (e.g., the minimum intra-cluster similarity or the desired number of clusters). In this paper, we propose a parallelised hierarchical clustering approach for categorical data (PH-clustering) using vertical data structures. In order to minimise the impact of low support items, we devise a weighted confidence (WC) affinity function to compute the similarity between clusters. Based on our analysis of the major clustering steps, we adopt a partial local and partial global approach to reduce computation time as well as to keep network communication at minimum. Load balance issues are addressed especially during the data partitioning phase. Our experimental results on standardised market basket data show that the proposed weighted confidence affinity measure is more accurate than other contemporary affinity measures in the literature and that our parallel clustering approach provides magnitudes of time improvements over sequential clustering especially over larger data sizes. Our results also indicate that the number of items/attributes in the dataset has a more drastic impact on performance than the number of transactions/tuples. Inderscience Publishers - linking academia, business and industry through research

Title: Parallel hierarchical clustering using weighted confidence affinity

Authors: Baoying Wang, Imad Rahal, Aijuan Dong

Addresses: Waynesburg University, Waynesburg, PA 15370, USA. ' College of Saint Benedict, Saint John's University, Collegeville, MN 56321, USA. ' Hood College, Frederick, Maryland 21701, USA

Abstract: There have been many attempts for clustering categorical data such as market basket dataset. However, most of categorical clustering approaches belong to partitional clustering which requires at least one input parameter (e.g., the minimum intra-cluster similarity or the desired number of clusters). In this paper, we propose a parallelised hierarchical clustering approach for categorical data (PH-clustering) using vertical data structures. In order to minimise the impact of low support items, we devise a weighted confidence (WC) affinity function to compute the similarity between clusters. Based on our analysis of the major clustering steps, we adopt a partial local and partial global approach to reduce computation time as well as to keep network communication at minimum. Load balance issues are addressed especially during the data partitioning phase. Our experimental results on standardised market basket data show that the proposed weighted confidence affinity measure is more accurate than other contemporary affinity measures in the literature and that our parallel clustering approach provides magnitudes of time improvements over sequential clustering especially over larger data sizes. Our results also indicate that the number of items/attributes in the dataset has a more drastic impact on performance than the number of transactions/tuples.

Keywords: parallel clustering; hierarchical clustering; weighted confidence affinity; message passing interface; market basket data; data mining; data partitioning; parallel computing; vertical data structures; categorical clustering; load balancing.

DOI: 10.1504/IJDMMM.2011.041491

International Journal of Data Mining, Modelling and Management, 2011 Vol.3 No.2, pp.110 - 129

Published online: 26 Feb 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Parallel hierarchical clustering using weighted confidence affinity

Keep up-to-date