Use of Propensity Scores in Non-Linear Response Models: The Case for Health Care Expenditures

@article{citeulike:2944670, abstract = {Author contact info: Anirban Basu Section of General Internatl Medicine Department of Medicine University of Chicago 5841 S. Maryland Avenue MC-2007, AMD B226C Chicago, IL 60637 Tel: 773/834-1796 Fax: 773/834-2238 E-Mail: abasu@medicine.bsd.uchicago.edu Daniel Polsky University of Pennsylvania School of Medicine Division of General Internal Medicine 423 Guardian Drive, Blockley Hall, Rm 1212 Philadelphia, PA 19104 E-Mail: polsky@mail.med.upenn.edu Willard G. Manning University of Chicago Harris School of Public Policy Studies 1155 East 60th Street, Room 176 Chicago, IL 60637 E-Mail: w-manning@uchicago.edu Under the assumption of no unmeasured confounders, a large literature exists on methods that can be used to estimating average treatment effects (ATE) from observational data and that spans regression models, propensity score adjustments using stratification, weighting or regression and even the combination of both as in doubly-robust estimators. However, comparison of these alternative methods is sparse in the context of data generated via non-linear models where treatment effects are heterogeneous, such as is in the case of healthcare cost data. In this paper, we compare the performance of alternative regression and propensity score-based estimators in estimating average treatment effects on outcomes that are generated via non-linear models. Using simulations, we find that in moderate size samples (n= 5000), balancing on estimated propensity scores balances the covariate means across treatment arms but fails to balance higher-order moments and covariances amongst covariates, raising concern about its use in non-linear outcomes generating mechanisms. We also find that besides inverse-probability weighting (IPW) with propensity scores, no one estimator is consistent under all data generating mechanisms. The IPW estimator is itself prone to inconsistency due to misspecification of the model for estimating propensity scores. Even when it is consistent, the IPW estimator is usually extremely inefficient. Thus care should be taken before naively applying any one estimator to estimate ATE in these data. We develop a recommendation for an algorithm which may help applied researchers to arrive at the optimal estimator. We illustrate the application of this algorithm and also the performance of alternative methods in a cost dataset on breast cancer treatment.}, author = {Basu, Anirban and Polsky, Daniel and Manning, Willard G. }, citeulike-article-id = {2944670}, journal = {National Bureau of Economic Research Working Paper Series}, keywords = {evaluation, matching, propensity-score, treatment}, month = {June}, pages = {14086+}, posted-at = {2008-06-30 14:53:25}, priority = {3}, title = {Use of Propensity Scores in Non-Linear Response Models: The Case for Health Care Expenditures}, url = {http://www.nber.org/papers/w14086}, year = {2008} }

TY - JOUR ID - citeulike:2944670 L3 - citeulike-article-id:2944670 TI - Use of Propensity Scores in Non-Linear Response Models: The Case for Health Care Expenditures JF - National Bureau of Economic Research Working Paper Series SP - 14086 N2 - Author contact info: Anirban Basu Section of General Internatl Medicine Department of Medicine University of Chicago 5841 S. Maryland Avenue MC-2007, AMD B226C Chicago, IL 60637 Tel: 773/834-1796 Fax: 773/834-2238 E-Mail: abasu@medicine.bsd.uchicago.edu Daniel Polsky University of Pennsylvania School of Medicine Division of General Internal Medicine 423 Guardian Drive, Blockley Hall, Rm 1212 Philadelphia, PA 19104 E-Mail: polsky@mail.med.upenn.edu Willard G. Manning University of Chicago Harris School of Public Policy Studies 1155 East 60th Street, Room 176 Chicago, IL 60637 E-Mail: w-manning@uchicago.edu Under the assumption of no unmeasured confounders, a large literature exists on methods that can be used to estimating average treatment effects (ATE) from observational data and that spans regression models, propensity score adjustments using stratification, weighting or regression and even the combination of both as in doubly-robust estimators. However, comparison of these alternative methods is sparse in the context of data generated via non-linear models where treatment effects are heterogeneous, such as is in the case of healthcare cost data. In this paper, we compare the performance of alternative regression and propensity score-based estimators in estimating average treatment effects on outcomes that are generated via non-linear models. Using simulations, we find that in moderate size samples (n= 5000), balancing on estimated propensity scores balances the covariate means across treatment arms but fails to balance higher-order moments and covariances amongst covariates, raising concern about its use in non-linear outcomes generating mechanisms. We also find that besides inverse-probability weighting (IPW) with propensity scores, no one estimator is consistent under all data generating mechanisms. The IPW estimator is itself prone to inconsistency due to misspecification of the model for estimating propensity scores. Even when it is consistent, the IPW estimator is usually extremely inefficient. Thus care should be taken before naively applying any one estimator to estimate ATE in these data. We develop a recommendation for an algorithm which may help applied researchers to arrive at the optimal estimator. We illustrate the application of this algorithm and also the performance of alternative methods in a cost dataset on breast cancer treatment. KW - evaluation KW - matching KW - propensity-score KW - treatment AU - Basu, Anirban AU - Polsky, Daniel AU - Manning, Willard PY - 2008/June// UR - http://www.nber.org/papers/w14086 ER -