Title: Unbalanced sample size effect on genome-wide population differentiation studies

Authors: Kyunghee Han; Kyee-Zu Kim; Jung Mi Oh; In-Wha Kim; Kyungim Kim; Taesung Park

Addresses: Department of Statistics, Seoul National University, Gwanak-gu, Seoul 151-747, Korea ' Interdisciplinary Program in Bioinformatics, Seoul National University, Gwanak-gu, Seoul 151-747, Korea ' College of Pharmacy, Seoul National University, Gwanak-gu, Seoul 151-747, Korea ' College of Pharmacy, Seoul National University, Gwanak-gu, Seoul 151-747, Korea ' College of Pharmacy, Seoul National University, Gwanak-gu, Seoul 151-747, Korea ' Department of Statistics, Seoul National University, Interdisciplinary Program in Bioinformatics, Gwanak-gu, Seoul 151-747, Korea

Abstract: The fixation index (FST) is one of the most widely used measurements of genetic distance between populations. The data set from the international HapMap project has been served as a reference data set for population differentiation studies. FST is commonly used in order to compare the sample data with HapMap data. In this study, however, we show that the use of FST without consideration of sample sizes may mislead the result. In particular, we first demonstrate that FST suffers from imbalance of sample sizes through simulation studies and through the analysis of a large scale Korean genome-wide association data. Then, we propose a modified version of FST which is shown to be more robust to imbalance of sample size. In addition, the chi-square test commonly used for homogeneity test is shown to perform similarly to the modified version of FST.

Keywords: Bayes estimator; fixation index; genetic distance; genome-wide association study; GWAS; population differentiation; unbalanced sample size; bioinformatics.

DOI: 10.1504/IJDMB.2012.049303

International Journal of Data Mining and Bioinformatics, 2012 Vol.6 No.5, pp.490 - 504

Received: 10 May 2011
Accepted: 11 May 2011

Published online: 17 Dec 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article