Article: SMART: a subspace clustering algorithm that automatically identifies the appropriate number of clusters Journal: International Journal of Data Mining, Modelling and Management (IJDMMM) 2009 Vol.1 No.2 pp.149 - 177 Abstract: This paper presents a subspace k-means clustering algorithm for high-dimensional data with automatic selection of k. A new penalty term is introduced to the objective function of the fuzzy k-means clustering process to enable several clusters to compete for objects, which leads to merging some cluster centres and the identification of the |true| number of clusters. The algorithm determines the number of clusters in a dataset by adjusting the penalty term factor. A subspace cluster validation index is proposed and employed to verify the subspace clustering results generated by the algorithm. The experimental results from both the synthetic and real data have demonstrated that the algorithm is effective in producing consistent clustering results and the correct number of clusters. Some real datasets are used to demonstrate how the proposed algorithm can determine interesting sub-clusters in the datasets. Inderscience Publishers - linking academia, business and industry through research

Title: SMART: a subspace clustering algorithm that automatically identifies the appropriate number of clusters

Authors: Liping Jing, Junjie Li, Michael K. Ng, Yiu-ming Cheung, Joshua Huang

Addresses: Institute for Computational Mathematics and Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong. ' Institute for Computational Mathematics and Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong. ' Institute for Computational Mathematics and Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong. ' Department of Computer Science, Hong Kong Baptist University, Kowloon Tong, Hong Kong. ' E-Business Technology Institute, The University of Hong Kong, Pokfulam Road, Hong Kong

Abstract: This paper presents a subspace k-means clustering algorithm for high-dimensional data with automatic selection of k. A new penalty term is introduced to the objective function of the fuzzy k-means clustering process to enable several clusters to compete for objects, which leads to merging some cluster centres and the identification of the |true| number of clusters. The algorithm determines the number of clusters in a dataset by adjusting the penalty term factor. A subspace cluster validation index is proposed and employed to verify the subspace clustering results generated by the algorithm. The experimental results from both the synthetic and real data have demonstrated that the algorithm is effective in producing consistent clustering results and the correct number of clusters. Some real datasets are used to demonstrate how the proposed algorithm can determine interesting sub-clusters in the datasets.

Keywords: data mining; subspace clustering; fuzzy k-means; cluster numbers; weighting; high-dimensional data.

DOI: 10.1504/IJDMMM.2009.026074

International Journal of Data Mining, Modelling and Management, 2009 Vol.1 No.2, pp.149 - 177

Published online: 26 May 2009 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: SMART: a subspace clustering algorithm that automatically identifies the appropriate number of clusters

Keep up-to-date