Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach

Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44-0.71) to 0.69 (95% CI 0.56-0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.