Operationalizing implementation science frameworks to plan a hybrid effectiveness-implementation study of a digital health intervention

Background: Evaluating implementation of digital health interventions (DHIs) in practice settings is complex, involving diverse users and multistep processes. Proactive planning can ensure desired implementation determinants and outcomes are captured for hybrid studies, but operational guidance for DHI studies is limited.

Methods: We planned a cluster randomized, type II hybrid effectiveness-implementation trial testing PositiveLinks, a smartphone application for HIV care, compared to usual care (n = 6 sites per arm), among HIV outpatient sites in the DC Cohort Longitudinal HIV Study in Washington, DC. Our process included: 1) Defining components of the DHI and associated implementation strategy, 2) Selecting implementation science frameworks to accomplish evaluation aims, 3) Mapping framework dimensions, domains, and constructs to implementation strategy steps, 4) Modifying/creating instruments to collect data for implementation outcome measures and determinants and 5) Developing a compatible implementation science data collection and management plan.

Results: Specification of components of the DHI and implementation strategy identified relevant platform usage data and necessary implementer actions, toward planning measurement of program reach and adoption. A priori mapping of implementation strategy steps to the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework identified how data would be collected for each step/corresponding outcome measure. The multi-site cohort study provided infrastructure for prospective, scaled implementation research, including site research assistants (RAs) coordinating implementation. Existing cohort tools (periodic site assessments, patient consent logs) were adapted for the evaluation to further capture representativeness and reach/adoption 'denominators.' New survey instruments allowed for framework-guided evaluation of provider adaptations/dose/fidelity to planned implementation across a large number of participants and multiple timepoints. Some aspects of real-world implementation were challenging to mirror within the planned hybrid trial (e.g. RAs selected as de facto site implementation leads) or were modified to preserve internal validity of effectiveness measurement (e.g. PositiveLinks 'community of practice').

Conclusions: Challenges and opportunities arose in planning the implementation evaluation for PositiveLinks within a hybrid trial in a cohort population. Prospective hybrid trial planning must balance generalizability of implementation processes to 'real world' conditions, with rigorous trial procedures to measure intervention effectiveness. Rapid, scalable tools require further study to enable evaluations within large, multi-site hybrid studies.