skip to main content
10.1145/3459930.3469528acmconferencesArticle/Chapter ViewAbstractPublication PagesbcbConference Proceedingsconference-collections
research-article

Synthesized difference in differences

Published: 01 August 2021 Publication History

Abstract

We consider estimating the conditional average treatment effect for everyone by eliminating confounding and selection bias. Unfortunately, randomized clinical trials (RCTs) eliminate confounding but impose strict exclusion criteria that prevent sampling of the entire clinical population. Observational datasets are more inclusive but suffer from confounding. We therefore analyze RCT and observational data simultaneously in order to extract the strengths of each. Our solution builds upon Difference in Differences (DD), an algorithm that eliminates confounding from observational data by comparing outcomes before and after treatment administration. DD requires a parallel slopes assumption that may not apply in practice when confounding shifts across time. We instead propose Synthesized Difference in Differences (SDD) that infers the correct (possibly non-parallel) slopes by linearly adjusting a conditional version of DD using additional RCT data. The algorithm achieves state of the art performance across multiple synthetic and real datasets even when the RCT excludes the majority of patients.

References

[1]
Alberto Abadie. 2005. Semiparametric difference-in-differences estimators. The Review of Economic Studies 72, 1 (2005), 1--19.
[2]
Carlos Blanco, Nicolas Hoertel, Silvia Franco, Mark Olfson, Jian-Ping He, Saioa López, Ana González-Pinto, Frédéric Limosin, and Kathleen R Merikangas. 2017. Generalizability of clinical trial results for adolescent major depressive disorder. Pediatrics 140, 6 (2017).
[3]
David Card and Alan B Krueger. 1993. Minimum wages and employment: A case study of the fast food industry in New Jersey and Pennsylvania. Technical Report. National Bureau of Economic Research.
[4]
Judith Droitcour, George Silberman, and Eleanor Chelimsky. 1993. Cross-design synthesis: a new form of meta-analysis for combining results from randomized clinical trials and medical-practice databases. International Journal of Technology Assessment in Health Care 9, 3 (1993), 440--449.
[5]
Robert Hable. 2012. Asymptotic normality of support vector machine variants and other regularized kernel methods. Journal of Multivariate Analysis 106 (2012), 92--117.
[6]
Jinyong Hahn. 1998. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica (1998), 315--331.
[7]
James J Heckman, Hidehiko Ichimura, and Petra Todd. 1998. Matching as an econometric evaluation estimator. The review of economic studies 65, 2 (1998), 261--294.
[8]
James J Heckman, Hidehiko Ichimura, and Petra E Todd. 1997. Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. The review of economic studies 64, 4 (1997), 605--654.
[9]
Maximilian Ilse, Patrick Forré, Max Welling, and Joris M Mooij. 2021. Efficient Causal Inference from Combined Observational and Interventional Data through Causal Reductions. arXiv preprint arXiv:2103.04786 (2021).
[10]
Rauf Izmailov, Vladimir Vapnik, and Akshay Vashist. 2013. Multidimensional splines with infinite number of knots as SVM kernels. In The 2013 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--7.
[11]
Christopher Jackson, John Stevens, Shijie Ren, Nick Latimer, Laura Bojke, Andrea Manca, and Linda Sharples. 2017. Extrapolating survival from randomized trials using external data: a review of methods. Medical Decision Making 37, 4 (2017), 377--390.
[12]
Nathan Kallus, Aahlad Manas Puli, and Uri Shalit. 2018. Removing Hidden Confounding by Experimental Grounding. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc., 10888--10897. https://proceedings.neurips.cc/paper/2018/file/566f0ea4f6c2e947f36795c8f58ba901-Paper.pdf
[13]
Jared K Lunceford and Marie Davidian. 2004. Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Statistics in medicine 23, 19 (2004), 2937--2960.
[14]
Joseph P McEvoy, Jonathan M Meyer, Donald C Goff, Henry A Nasrallah, Sonia M Davis, Lisa Sullivan, Herbert Y Meltzer, John Hsiao, T Scott Stroup, and Jeffrey A Lieberman. 2005. Prevalence of the metabolic syndrome in patients with schizophrenia: baseline results from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia trial and comparison with national estimates from NHANES III. Schizophrenia research 80, 1 (2005), 19--32.
[15]
Judea Pearl. 2009. Causality. Cambridge university press.
[16]
Paul R Rosenbaum and Donald B Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 1 (1983), 41--55.
[17]
Evan Rosenman, Guillaume Basse, Art Owen, and Michael Baiocchi. 2020. Combining observational and experimental datasets using shrinkage estimators. arXiv preprint arXiv:2002.06708 (2020).
[18]
Donald B Rubin. 1978. Bayesian inference for causal effects: The role of randomization. The Annals of statistics (1978), 34--58.
[19]
A John Rush, Madhukar H Trivedi, Stephen R Wisniewski, Andrew A Nierenberg, Jonathan W Stewart, Diane Warden, George Niederehe, Michael E Thase, Philip W Lavori, Barry D Lebowitz, et al. 2006. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR* D report. American Journal of Psychiatry 163, 11 (2006), 1905--1917.
[20]
Linmarie Sikich, Jean A Frazier, Jon McClellan, Robert L Findling, Benedetto Vitiello, Louise Ritz, Denisse Ambler, Madeline Puglia, Ann E Maloney, Emily Michael, et al. 2008. Double-blind comparison of first-and second-generation antipsychotics in early-onset schizophrenia and schizo-affective disorder: findings from the treatment of early-onset schizophrenia spectrum disorders (TEOSS) study. American Journal of Psychiatry 165, 11 (2008), 1420--1431.
[21]
T Scott Stroup, Jeffrey A Lieberman, Joseph P McEvoy, Sonia M Davis, Marvin S Swartz, Richard SE Keefe, Alexander L Miller, Robert A Rosenheck, John K Hsiao, CATIE Investigators, et al. 2009. Results of phase 3 of the CATIE schizophrenia trial. Schizophrenia research 107, 1 (2009), 1--12.
[22]
Sofia Triantafillou and Gregory Cooper. 2020. Learning Adjustment Sets from Observational and Limited Experimental Data. arXiv preprint arXiv:2005.08749 (2020).
[23]
Sofia Triantafillou, Fattaneh Jabbari, and Greg Cooper. 2021. Causal Markov Boundaries. arXiv preprint arXiv:2103.07560 (2021).
[24]
Madhukar H Trivedi, A John Rush, Stephen R Wisniewski, Andrew A Nierenberg, Diane Warden, Louise Ritz, Grayson Norquist, Robert H Howland, Barry Lebowitz, Patrick J McGrath, et al. 2006. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR* D: implications for clinical practice. American journal of Psychiatry 163, 1 (2006), 28--40.
[25]
Diane Warden, A John Rush, Madhukar H Trivedi, Maurizio Fava, and Stephen R Wisniewski. 2007. The STAR* D Project results: a comprehensive review of findings. Current psychiatry reports 9, 6 (2007), 449--459.
[26]
Shuxi Zeng, Murat Ali Bayir, Joseph J Pfeiffer III, Denis Charles, and Emre Kiciman. 2021. Causal transfer random forest: Combining logged data and randomized experiments for robust prediction. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 211--219.

Cited By

View all
  • (2024)Improving the Care of Severe, Open Fractures and Postoperative Infections of the Lower Extremities: Protocol for an Interdisciplinary Treatment ApproachJMIR Research Protocols10.2196/5782013(e57820)Online publication date: 16-Sep-2024
  • (2023)The Policy Impact of Carbon Emission Trading on Building Enterprises’ Total Factor Productivity in ChinaBuildings10.3390/buildings1306149313:6(1493)Online publication date: 9-Jun-2023

Index Terms

  1. Synthesized difference in differences
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        BCB '21: Proceedings of the 12th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics
        August 2021
        603 pages
        ISBN:9781450384506
        DOI:10.1145/3459930
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 01 August 2021

        Permissions

        Request permissions for this article.

        Check for updates

        Badges

        • Best Paper

        Author Tags

        1. causal inference
        2. cross design synthesis
        3. difference in differences

        Qualifiers

        • Research-article

        Conference

        BCB '21
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 254 of 885 submissions, 29%

        Upcoming Conference

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)7
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 10 Nov 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Improving the Care of Severe, Open Fractures and Postoperative Infections of the Lower Extremities: Protocol for an Interdisciplinary Treatment ApproachJMIR Research Protocols10.2196/5782013(e57820)Online publication date: 16-Sep-2024
        • (2023)The Policy Impact of Carbon Emission Trading on Building Enterprises’ Total Factor Productivity in ChinaBuildings10.3390/buildings1306149313:6(1493)Online publication date: 9-Jun-2023

        View Options

        Get Access

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media