Article: Is matching in different situations equally applicable for impact evaluation studies when using observational data? Journal: International Journal of Data Analysis Techniques and Strategies (IJDATS) 2022 Vol.14 No.1 pp.55 - 78 Abstract: Under certain circumstances, randomisation of interventions may not be applicable in development activities. Quasi-experimental research design such as matching often helps creating counterfactuals for impact evaluations. Although different types of statistical matching methods are available, their relative performance is generally unknown to practitioners. Using five sets of household survey data collected from samples of treatment and comparison groups from four countries in the Hindu Kush Himalaya region, we examine the extent of covariate imbalances before and after matching using four different matching methods. For small samples with enough imbalances in the covariates, the nearest neighbour matching does not perform well, but matching with stratification works better. The performance of radius and kernel matching falls in between nearest neighbour and matching with stratification. We find that the matching is useful when covariates imbalance is high before matching but less useful for sample with relatively balanced covariates before matching. Inderscience Publishers - linking academia, business and industry through research

Title: Is matching in different situations equally applicable for impact evaluation studies when using observational data?

Authors: Apsara Karki Nepal; Ghulam Muhammad Shah; Farid Ahmad

Addresses: International Centre for Integrated Mountain Development (ICIMOD), P.O. Box 3226, Khumaltar, Lalitpur, Kathmandu, Nepal ' International Centre for Integrated Mountain Development (ICIMOD), P.O. Box 3226, Khumaltar, Lalitpur, Kathmandu, Nepal ' International Centre for Integrated Mountain Development (ICIMOD), P.O. Box 3226, Khumaltar, Lalitpur, Kathmandu, Nepal

Abstract: Under certain circumstances, randomisation of interventions may not be applicable in development activities. Quasi-experimental research design such as matching often helps creating counterfactuals for impact evaluations. Although different types of statistical matching methods are available, their relative performance is generally unknown to practitioners. Using five sets of household survey data collected from samples of treatment and comparison groups from four countries in the Hindu Kush Himalaya region, we examine the extent of covariate imbalances before and after matching using four different matching methods. For small samples with enough imbalances in the covariates, the nearest neighbour matching does not perform well, but matching with stratification works better. The performance of radius and kernel matching falls in between nearest neighbour and matching with stratification. We find that the matching is useful when covariates imbalance is high before matching but less useful for sample with relatively balanced covariates before matching.

Keywords: matching; variable imbalance; impact evaluation; propensity scores; comparison group; observational data.

DOI: 10.1504/IJDATS.2022.121512

International Journal of Data Analysis Techniques and Strategies, 2022 Vol.14 No.1, pp.55 - 78

Received: 06 Jan 2021
Accepted: 06 Jul 2021
Published online: 16 Mar 2022 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Is matching in different situations equally applicable for impact evaluation studies when using observational data?

Keep up-to-date