Conferenc e Proceedings

Implementation Science 2020, 15(Suppl 2):80
https://doi.org/10.1186/s13012-020-01033-8
MEETING ABSTRACTS Open Access
Proceedings of the Fifth Biennial

Conference of the Society for
Implementation Research Collaboration
(SIRC) 2019: where the rubber meets the
road: the intersection of research, policy,
and practice - part 2
Seattle, WA, USA. 12-14 September 2019
Published: 30 September 2020
wants their decisions to be supported by evidence, this information

About this supplement could spur policymakers to make more evidence-informed health pol-
icy decisions and demonstrate evidence use to their constituents. How-
ever, no prior research has examined public opinion about evidence-
This article has been published as part of informed policymaking. This study sought to characterize public opin-
Implementation Science Volume 15 Supplement 2, ion about the influence that evidence should, and does, have on health
policy development in U.S. Congress relative to other factors and exam-
2020: Proceedings of the Fifth Biennial Conference of ine differences in opinion by political party affiliation.
the Society for Implementation Research Materials and Methods
Collaboration (SIRC) 2019: Where the rubber meets A public opinion survey was conducted in 2018 using the SSRS Prob-
ability Panel (N=532), a nationally representative internet panel. Re-
the road: The intersection of research, policy, and spondents separately rated the extent to which six factors (e.g.,
practice - Part 2. The first part of this supplement is evidence, budget impact, industry interests) “should have” and “cur-
available online at https://implementationscience. rently have” influence on U.S. congresspersons’ health policy decisions.
Results
biomedcentral.com/articles/supplements/volume-15- Evidence (59%) was the most frequently identified factor that should
supplement-3. Please note that this is part 2 of 2. have “a lot of influence” on health policy development, but only 11%
of respondents thought that evidence currently has “a lot of influ-
A1 ence” (p<.001). Opinions about evidence did not vary significantly by
Public opinion as an outer-contextual factor in health policy D&I respondent political party affiliation.
research and practice: evidence that the public cares about Conclusions
evidence There is strong bi-partisan public support for evidence to have much
Correspondence: Jonathan Purtle ([email protected]) more influence on health policymaking in U.S. Congress. This finding
Dornsife School of Public Health, Drexel University, Philadelphia, PA, USA is promising in a time of elevated political polarization in the United
Implementation Science 2020, 15(Suppl 2):A1 States. The survey results have implications for interventions that aim
to promote evidence-informed health policymaking. As findings sug-
Background gest public demand for evidence-informed health policymaking, and
Barriers to evidence-informed health policymaking are well-established prior research demonstrating that public opinion often influences
[1]. Although many barriers are technical in nature (e.g., poor communi- elected policymakers’ behaviors [5-6], interventions that systematic-
cation of research findings) [2], a major impediment stems from the ally document the extent to which elected policymakers’ actions
political nature of policymaking [3-4]. Public opinion is a key aspect of (e.g., public statements, content of bills introduced, tweets) are
politics; and one that is relevant to efforts to promote evidence- evidence-supported and disseminate this information to the public
informed policymaking because public opinion influences policy- could encourage policymakers to make more evidence-informed
makers’ behaviors [5-6]. Thus, if policymakers learn that the public health policy decisions.
© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article's Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.
Implementation Science 2020, 15(Suppl 2):80 Page 2 of 85
References 3. Haskins R, Margolis G. Show me the evidence: Obama’s fight for rigor and
1. National Research Council. Using science as evidence in public policy. evidence in social policy. Washington, DC: Brookings Institution Press; 2015.
Schwandt TA, Straf ML, editors. Washington, DC: The National Academies 4. Baron J, Haskins R. The Obama administration’s evidence-based social
Press; 2012. doi:https://doi.org/10.17226/13460. policy initiatives: An overview. 2011. http://www.brookings.edu/opinions/
2. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review 2011/1102_budget_deficit_haskins.aspx.
of barriers to and facilitators of the use of evidence by policymakers. 5. Lynch JP, Sabol WJ. Assessing the effects of mass incarceration on
BMC Health Serv Res. 2014;14(1):2. informal social control in communities. Criminol Public Pol. 2004;3(2):267-
3. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences 294. doi:10.1111/j.1745-9133.2004.tb00042.x
on the use of evidence in public health policy: a systematic review. PloS 6. Ratzliff A, Phillips KE, Sugarman JR, Unützer J, Wagner EH. Practical
One. 2013;8(10):e77404. approaches for achieving integrated behavioral health care in primary
4. Oliver TR. The politics of public health policy. Annu Rev Public Health. care settings. Am J Med Qual. 2017;32(2):117-121. doi:10.1177/
2006;27:195-233. 1062860615618783
5. Burstein P. The impact of public opinion on public policy: a review and 7. Crowley M, Scott T, Fishbein DH. Translating prevention research for
an agenda. Polit Research Q. 2003;56(1):29-40. evidence-based policymaking: Results from the Research-to-Policy Collab-
6. Butler DM, Nickerson DW. Can learning constituency opinion affect how oration pilot. Prev Sci. 2018;19(2):260-270.
legislators vote? Results from a field experiment. Q J Polit Sci.
2011;6(1):55-83.
A3
Measurement infrastructure for influencing the outer context:
A2 Integrating evidence-based practice reporting and client surveys
Federal mental health legislation: what becomes law and why? to guide decision-making in a learning health care system
Noah R. Gubner1,2, Felix I. Rodriguez3, Rose Krebill-Prather4, Kristen
Results from a 30-year review
Petersen4, Sarah Cusworth Walker1,2
Correspondence: Max Crowley ([email protected]) 1
College of Health and Human Development, Penn State University, Department of Psychiatry and Behavioral Sciences, University of
University Park, PA, USA Washington, Seattle, WA, USA; 2Washington State Evidence-Based
Practice Institute, Seattle, WA, USA; 3Washington State Health Care
Implementation Science 2020, 15(Suppl 2):A2
Authority, Olympia, WA, USA; 4Social and Economic Sciences Research
Background Center, Washington State University, Pullman, WA, USA
Mental health problems affect millions of individuals and cost over Correspondence: Sarah Cusworth Walker ([email protected])
$240 billion annually in increased healthcare, criminal justice, child
welfare, education, and labor costs [1]. Ongoing efforts to take
evidence-based mental health strategies to scale have encountered a Background
number of barriers to successful uptake and sustained use [2-4]. To Legislation in Washington State (HB2536), passed in 2012, mandated
the reporting of evidence-based practices (EBPs) for all children-serving
overcome these barriers, researchers and advocates often seek to in-
systems in Washington State. In response, the Division of Behavioral
fluence public policy to facilitate increased investment in mental
health [5-7]. Yet, little work has systematically sought to understand Health and Recovery and University of Washington’s Evidence-Based
the specific content of past mental health legislation and how the Practice Institute developed a measurement method to track use of
EBPs in routine services statewide. This method provides a cost effect-
content of the legislation is related to whether a bill becomes law.
ive and adaptable surveillance tool to monitor evidence-based prac-
Materials and Methods
In order to answer these pressing questions about mental health pol- tices (EBPs) and can also be merged with other data sources to monitor
icy, we conducted a mixed methods review of all federal bills intro- disparities in utilization and effectiveness. In this paper we will present
this as an example of “outer setting” influences on EBP use as well as
duced to Congress over the last three decades (1989-2019; N=
proof of concept for this integrated data infrastructure as a means to
171,861). This includes systematic coding of mental health’s inclusion
in federal legislation, quantitative analyses of that inclusion’s relation- guide decision-making at the policy level.
ship with bills becoming law, and qualitative analyses of how mental Materials and Methods
We analyzed a sample of youth (<21 years old) from Washington State
health policy may be used to improve population health.
who had received at least one hour of publicly funded outpatient men-
Results
In the 101st Congress (January 3, 1989, to January 3, 1991), only 14 tal health (MH) service. Current procedural terminology (CPT) billing
mental health bills were introduced, comprising only .001% of all bills codes were used to identify if clients in the sample received any valid
EBP psychotherapy sessions during the observation period (May – Oct
introduced. By the 115th Congress (January 3, 2017, to January 3,
2015). Billing data were then tied to self-reported perceptions of
2019), over 4% of all bills of any type included mental health provisions.
In addition to increases in mental health policy across time, we will also outcomes of service from the Child-Family Mental Health Consumer
present results of analyses that identify characteristics of bills that are Survey. These outcome data were compared between youth who did
and did not receive a valid EBP session during the observation period
more likely to be successfully enacted into law. Finally, results from
Results
qualitative analyses will be used to illustrate how elements of mental
health policies can facilitate or restrict high-quality implementation. Among this sample of 1,580 youth, 19.7% (n=312) received at least
Conclusions one valid EBP psychotherapy session. Youth from rural (21.4%) versus
urban (16.2%) providers were more likely to have received an EBP
We will discuss implications of these findings through the lens of the
session (χ2=6.02, p=0.014). There were significant differences by
outer context of the implementation ecology; specifically, by identify-
ing supports and barriers from federal legislation and policy that race/ethnicity (χ2=14.71, p=0.04). Non-Hispanic Whites (20.2%) and
may be most likely to promote successful implementation of American Indians (33.3%) were more likely, while African Americans
(12.3%) and Hispanics (15.7%) were less likely to have received a
evidence-based treatments. We will also discuss implications for how
valid EBP session. There was a trend for an interaction between race/
best to engage directly with policymakers to support increased avail-
ability and effectiveness of mental health services. ethnicity and receipt of an EBP session on self-reported positive
services outcomes, with receipt of an EBP session potentially being
associated with more positive services outcomes among youth who
References were non-White versus White.
1. O’Connell ME, Boat TF, Warner KE. Preventing mental, emotional, and Conclusions
behavioral disorders among young people: progress and possibilities. As a proof of concept, we demonstrate that billing data provides
Washington, D.C.: National Academies Press; 2009. a cost-effective tool to monitor the receipt of EBP MH sessions
2. CDC. The Power of Prevention. Atlanta, GA: Centers for Disease Control and can be tied to other data sources to examine outcomes
and Prevention; 2009. across a health network.
A4 A5
Giving the outer setting its due: adapting the stages of Cross-collaborations among researchers, community, government
implementation completion to policy and system-level change efforts agencies, and a federal funding agency to support implementation
Eric J. Bruns1, Jonathan R. Olson1, Philip H. Benjamin1, Lisa Saldana2 of evidence-based cardiovascular disease prevention in primary
1
Department of Psychiatry & Behavioral Sciences, University of Washington, care: the EvidenceNOW Initiative
Seattle, WA, USA; 2Oregon Social Learning Center, Eugene, OR, USA Donna Shelley1, Michael Parchman2, Robert McNellis3
1
Correspondence: Eric J. Bruns ([email protected]) School of Medicine, New York University, New York, NY, USA; 2Kaiser
Implementation Science 2020, 15(Suppl 2):A4 Permanente Washington Health Research Institute, Seattle, WA, USA;
3
Agency for Healthcare Research and Quality, Rockville, MD, USA
Background Correspondence: Donna Shelley ([email protected])
Successful implementation of Wraparound care coordination for youth Implementation Science 2020, 15(Suppl 2):A5
with complex behavioral health needs requires hospitable policy and fi-
nancing conditions [1]. Thus, when the “rubber meets the road,” imple- Background
mentation support for Wraparound requires attention to the “outer EvidenceNOW is an Agency for Healthcare Research and Quality (AHRQ)-
setting” of the implementation ecology [2], including cross-agency coord- funded research initiative focused on helping primary care practices im-
ination, Medicaid payment reform, and cross-sector information systems. prove the delivery of the “ABCS” of cardiovascular disease (CVD) preven-
Unfortunately, existing implementation measures are scarce at the outer tion: Aspirin in high-risk individuals, Blood pressure control, Cholesterol
context [3], as is research on relevant outer setting strategies [4]. This management, and Smoking cessation. Seven cooperatives used practice
presentation describes efforts to track Wraparound implementation at facilitation (PF) as a unifying strategy to support the implementation and
system and organizational levels using an adaptation of the Stages of Im- dissemination of evidence-based care for CVD risk factors. Each coopera-
plementation Completion (SIC) [5]. tive enrolled over 200 practices in their region. Cooperatives created an
Materials and Methods infrastructure to engage stakeholder organizations in their region to facili-
The SIC assesses implementation across eight stages and three phases: pre- tate the initiative. AHRQ created infrastructure that included a national
implementation, implementation, and sustainment. We adapted and added evaluator (ESCALATES) and committees and meetings that fostered
SIC items to align with multilevel Wraparound implementation support as cross-cooperative collaboration and dissemination.
enacted by the National Wraparound Implementation Center (NWIC), includ- Materials and Methods
ing an array of state systems (outer setting) variables such as state leadership This panel includes an AHRQ program officer who will discuss the infrastruc-
engagement, cross-system communication and collaboration, financing ture created and lessons learned for guiding the design of future funding
strategies, and contract requirements. We have collected data for two pilot opportunities [1] and case studies demonstrating how two cooperatives,
states and are completing collation for eight additional states. HealthyHearts New York City [2] and Healthy Hearts Northwest [3], partnered
Results with community and government agencies in their region to accomplish
Adaptation of the SIC entailed adding 12 items and removing 4. Further- EvidenceNOW goals. The discussant is the ESCALATES Principal Investigator.
more, we operationalized each SIC variable in terms of measurable Wrap- Results
around processes and activities. Preliminary results from two pilot states The aligned efforts of Cooperatives, AHRQ, and ESCALATES facilitated the
indicate high completion rates across stages for both states (completion collective expansion of capacity for practice transformation and rigorous
percentages from 60% to 100%). However, states differed significantly in evaluation. We found that: (a) large-scale, federally-funded research initia-
their time to completion, with State 1 averaging 3.75 months for comple- tives were strengthened through infrastructure that fostered collabor-
tion of stages and state 2 averaging 26.38 months. Item- and stage-level ation across grantees (demonstrated by 12 cross-cooperative
analyses revealed that State 2 struggled to engage state leadership in im- publications to date); (b) an external evaluator added tremendous value
plementation. State 1 adopted a new approach to building Wraparound in supporting cross-cooperative collaboration and amplified opportunities
implementation infrastructure – investing in Care Management Entities for dissemination; (c) engaging with partner organizations early helped
(CMEs) [6], while State 2 relied on Community Mental Health Centers. assess fit with organizational strategic plans, capacity, readiness for
Conclusions change, and data collection systems; (d) applying an implementation sci-
Findings provide proof of concept for incorporating outer setting ence framework [4] was necessary to guide intervention development
items into an established implementation measure and underscore and assessment; and (e) stakeholder organizations valued being included
the influence of outer context in Wraparound implementation. The in research and funder meetings and dissemination activities.
presentation will show how NWIC is using this measure to support Conclusions
states to build systems that facilitate better implementation and out- For cooperatives, aligning goals, and attending to opportunities to
comes, while also promoting new and needed research for mental grow research and quality improvement capacity among partnering
health and implementation science. agencies facilitated strong, enduring partnerships for practice trans-
formation. Funders have a role to play in facilitating collaborations
among cooperatives and evaluation. The findings that emerge through
References
the efforts of multi-level partners are greater than the sum of the parts
1. Bruns EJ, Sather A, Pullmann MD, Stambaugh LF. National trends in
and further the field of dissemination and implementation science.
implementing wraparound: results from the state wraparound survey. J
Child Fam Stud. 2011;20(6):726-735.
2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. References
Fostering implementation of health services research findings into 1. Cohen DJ, Balasubramanian BA, Gordon L, Marino M, Ono S, Solberg LI,
practice: a consolidated framework for advancing implementation Crabtree BF, Strange KC, Davis M, Miller WL, Damschroder LJ, McConnell
science. Implement Sci. 2009; 4:50. doi:10.1186/1748-5908-4-50. KJ, Creswell J. A national evaluation of a dissemination and
3. Lewis CC, Stanick CF, Martinez RG, Weiner BJ, Kim M, Barwick M, Comtois KA. The implementation initiative to enhance primary care practice capacity and
Society for Implementation Research Collaboration Instrument Review Project: a improve cardiovascular disease care: the ESCALATES study protocol.
methodology to promote rigorous evaluation. Implement Sci. 2015; 10:2. Implement Sci. 2016;11(86):1-13. doi:10.1186/s13012-016-0449-8
4. Purtle J, Peters R, Brownson RC. A review of policy dissemination and 2. Shelley DR, Ogedegbe G, Anane S, Wu WY, Goldfield K, Gold HT, Kaplan
implementation research funded by the National Institutes of Health, S, Berry C. Testing the use of practice facilitation in a cluster randomized
2007–2014. Implement Sci. 2016;11:1. stepped-wedge design trial to improve adherence to cardiovascular dis-
5. Saldana L, Chamberlain P, Wang W, Brown CH. Predicting program start- ease prevention guidelines: HealthyHearts NYC. Implement Sci.
up using the stages of implementation measure. Adm Policy Ment 2015;11(1):88. doi:10.1186/s13012-016-0450-2
Health. 2012:39(6):419-25. 3. Parchman ML, Fagnan LJ, Dorr DA, Evans P, Cook AJ, Penfold RB, Hsu C, Cheadle
6. Center for Health Care Strategies. Care management entities: a primer. A, Baldwin LM, Tuzzio L. Study protocol for “Healthy Hearts Northwest”: a 2 × 2
2011. https://www.chcs.org/resource/care-management-entities-a-primer/ randomized factorial trial to build quality improvement capacity in primary care.
. Accessed 27 March 2019. Implement Sci. 2016;11(138):1-9. doi:10.1186/s13012-016-0502-7
4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. opportunity to promote cross-discipline collaboration. Community Based
Fostering implementation of health services research findings into Learning Collaboratives (CBLC) use specific training/ implementation
practice: a consolidated framework for advancing implementation strategies involving multidisciplinary stakeholders to foster collaboration
science. Implement Sci. 2009;4(1):50. doi:10.1186/1748-5908-4-50 and build community capacity for trauma-focused EBTs [4]. The broker
curriculum includes information about trauma impact, trauma-focused
A6 EBTs, family engagement strategies, and trauma-focused treatment plan-
A collaboration between practitioners, intermediaries, and ning, while also providing opportunities for cross-discipline training.
researchers to increase access to evidence-based chronic pain care Materials and Methods
Jessica Chen1,2, Lisa Glynn2, Timothy Dawson1,2, Hannah Gelman3, This presentation examines changes in trauma-related knowledge,
Steven Zeliadt1,3 practices and interprofessional collaboration among n = 33 brokers
1
Department of Psychology, University of Washington, Seattle, WA, USA; participating in CBLCs conducted as part of a statewide dissemin-
2
VA Puget Sound, Seattle, WA, USA; 3Seattle-Denver Center of ation initiative. Brokers completed self-report measures examining
Innovation, VA HSR&D, Seattle, WA, USA knowledge of trauma-related topics taught as part of the CBLC cur-
Correspondence: Jessica Chen ([email protected]) riculum, organizational climate, interprofessional collaboration, and
Implementation Science 2020, 15(Suppl 2):A6 trauma-related practices (e.g., assessment, psychoeducation) pre- and
post-CBLC. Bivariate correlations were computed in SPSS, and inter-
Background action effects were probed using the PROCESS macro in SPSS.
This presentation describes initial lessons learned from a collaboration Results
between front-line providers, intermediaries, and researchers to imple- Analyses revealed that increases in knowledge about the trauma-related
ment a telehealth model of chronic pain care to rural VA clinics. curriculum topics were significantly associated with positive changes in
As opioid-related overdoses have increased exponentially in the U.S. over broker practices following CBLC participation. However, neither changes
the last two decades, there has been greater emphasis on increasing ac- related to interprofessional collaboration nor organizational climate were
cess to non-opioid and non-pharmacological pain management, particu- significantly related to changes in broker practices. Moderation analyses
larly to rural and under-served areas and in primary care settings. revealed a significant interaction effect between interprofessional collab-
Materials and methods oration and knowledge of evidence-based treatment planning pre- and
To address these priorities, front-line clinicians and clinical administra- post-CBLC [F (1, 29) = 5.14; p = .03; R2 = .25]. Those participants who re-
tors at one Veterans Health Administration (VA) facility initiated a col- ported the greatest pre- to post-CBLC change in interprofessional collab-
laboration with VA implementation researchers to implement a novel oration also reported a significant increase in skills related to evidence-
hub-and-spoke telehealth model for chronic pain management (Tele- based treatment planning (t = 2.62, p = .03, 95% CI = .04, 0.84).
Pain), which will deliver patient education, evidence-based psychother- Conclusions
apies, movement therapies (e.g., yoga for lower back pain), and non- Study findings suggest that brokers play an important role in building
opioid pharmacotherapies from a central specialty pain “hub” to rural community capacity for EBT access and that cross-discipline strategies
VA primary care clinic “spokes.” The goal of the collaboration between help to foster collaborative relationships among youth service providers.
practitioners, intermediaries, and researchers was to obtain grant fund- Implications for future research, policy and practice will be addressed.
ing to support external facilitation and evaluation activities.
Results References
There were several lessons learned from this collaborative process regard- 1. U.S. Department of Health & Human Services, Administration for Children
ing health care system changes that may better foster collaboration be- and Families, Administration on Children, Youth and Families, Children’s
tween research, policy, and practice. One was that the unpredictable and Bureau. Child maltreatment 2016. 2018. https://www.acf.hhs.gov/cb/
cyclical nature of grant funding can interfere with delivering implementa- research-data-technology/statistics-research/child-maltreatment
tion support when it is needed, namely at the time of clinical services roll- 2. Fitzgerald MM, Torres MM, Shipman K, Gorrono J, Kerns SEU, Dorsey S.
out. Therefore, when clinical programs are being designed, they should Child welfare caseworkers as brokers of mental health services: a pilot
consider building in funding for implementation support and evaluation evaluation of Project Focus Colorado. Child Maltreat. 2015; 20(1);37-49.
from the outset. A second lesson learned was that implementation inter- https://doi.org/10.1177/1077559514562448
mediaries, individuals who have experience with both care delivery and 3. Stiffman AR, Pescosolido B, Cabassa LJ. Building a model to understand
evaluative sciences, may be particularly important for bridging the cul- youth service access: the gateway provider model. Ment Health Serv Res.
tural divide between researchers and practitioners. 2004;6:189. https://doi.org/10.1023/B:MHSR. 0000044745.09952.33
Conclusions 4. Hanson RF, Saunders BE, Ralston E, Moreland AD, Peer SO, Fitzgerald MM.
Embedding intermediaries as fully-funded staff in the TelePain clinical Statewide implementation of trauma-specific treatment services using
program has helped to align research aims with clinical priorities and the Community-Based Learning Collaborative Model. Psychol Serv.
may support the long-term sustainability of implementation efforts. 2019;16:171-181.
A7 A8
Untangling trauma-related knowledge and practice changes Change in patient outcomes after augmenting a low-level
among brokers in a community-based learning collaborative: role implementation strategy in community practices that are slow to
of interprofessional collaboration adopt a collaborative chronic care model: a cluster randomized
Funlola Are1, Rochelle Hanson1, Samuel Peer2, Ben Saunders1 implementation trial
1
Department of Psychiatry and Behavioral Sciences, Medical University of Shawna Smith1, Daniel Almirall1, Katherine Prenovost1, Mark
South Carolina, Charleston, SC, USA; 2Psychology Department, Idaho Bauer2, Celeste Liebrecht1, Daniel Eisenberg1, Amy Kilbourne1
1
State University, Pocatello, ID, USA Institute for Healthcare Policy and Innovation University of Michigan,
Correspondence: Funlola Are ([email protected]) Ann Arbor, MI, USA; 2Department of Psychiatry, Harvard Medical School-
Implementation Science 2020, 15(Suppl 2):A7 VA Boston, Boston, MA, USA
Correspondence: Shawna Smith ([email protected])
Background Implementation Science 2020, 15(Suppl 2):A8
While evidence-based treatments (EBTs) exist to ameliorate trauma-
related mental health problems, many children do not receive them Background
[1]. Possible reasons to account for this include limited availability of Implementation strategies are essential for promoting uptake of
EBTs and poor collaboration amongst professionals involved in youth evidence-based practices and for patients to receive optimal care [1]. Yet
service provision [2]. Brokers, often child welfare workers, serve an im- strategies differ substantially in their intensity and feasibility. Lower-
portant intermediary role in improving service access for youth [3], but intensity strategies (e.g., training, technical support) are commonly used,
they are often trained separately from clinical providers, precluding the but may be insufficient for all clinics. Limited research has examined the
comparative effectiveness of augmentations to low-level implementation CCIs can be difficult to implement and efforts to scale up CCIs are often
strategies for non-responding clinics [2-3]. stymied by a lack of practitioner knowledge for identifying and addressing
Materials and methods barriers to implementation [1]. Implementation roadmaps provide a play-
In this Hybrid Type III implementation-effectiveness study [4], we com- book outlining critical steps practitioners should follow in scaling up CCIs
pare the effectiveness of two augmentation strategies, External Facilita- to new settings that overcome implementation barriers, measure imple-
tion (EF) vs. External + Internal Facilitation (EF/IF)[5] for improving uptake mentation success, and garnering leadership support for longer-term CCI
of an evidence-based collaborative care model (CCM) on 18-month men- sustainability [2]. We present an Implementation Roadmap [3] that guides
tal health outcomes for patients with depression at community-based community-based CCI implementation based on the experiences of stake-
clinics initially non-responsive to lower-level implementation support. holders successfully implementing a broad spectrum of CCIs through the
Results Michigan Mental Health Integration Partnership (MIP) [4]. The goal of MIP
Providers initially received support using a low-level implementation is to support the scale up and spread of CCIs that enhance access to care
strategy, Replicating Effective Programs (REP). After 6 months, non- for Medicaid-eligible consumers with behavioral healthcare needs, while
responsive clinics that had failed to deliver a clinically significant dose also providing an in-situ implementation laboratory for informing sus-
of the CCM to >10 patients were randomized to augment REP with ei- tained uptake of CCIs across a variety of community-based settings.
ther EF or EF/IF. Mixed effects models evaluated the comparative effect- Materials and Methods
iveness of the two augmentations on patient outcomes at 18 months. Semi-structured interviews were carried out with stakeholders from
The primary outcome was patient SF-12 mental health score; secondary successfully adopted MIP CCI projects to define common barriers, chal-
outcomes were PHQ-9 depression score and self-reported receipt of lenges, and implementation strategies deployed by the project teams.
the CCM during months 6 through 18. Interviews were transcribed and analyzed for common themes that
27 clinics were non-responsive after 6 months of REP. 13 clinics (N=77 pa- identified a series of critical steps scaffolding the CCI implementation
tients) were randomized to REP+EF and 14 (N=92) to REP+EF/IF. At 18 process and accompanying metrics for evaluating implementation
months, patients in the REP+EF/IF arm had worse SF-12 (diff=8.38; 95%CI= progress. 25 interviews of key stakeholders were conducted across 7
3.59, 13.18) and PHQ-9 scores (diff=1.82; 95%CI=-0.14, 3.79), and lower successful MIP implementation teams, including 11 providers at imple-
odds of CCM receipt (OR=0.67, 95% CI=0.30,1.49) than REP+EF patients. mentation sites and 14 researchers/project managers.
Conclusion Results
Patients at initially non-responsive community-based clinics that were Stakeholders commonly identified specific steps that overcame barriers
randomized to receive the more intensive EF/IF augmentation saw less to CCI implementation, including deployment of web-based tools for
improvement in mood symptoms at 18 months than those the re- facilitating implementation, embedding key metrics of implementation
ceived the EF augmentation, and were also no more likely to receive success, garnering upper level administration buy-in upfront, and speci-
the CCM. While EF generally appeared to help clinics, a number of EF/IF fying a process for tailoring implementation strategy deployment to
clinics experienced barriers in implementing the IF strategy with fidel- specific site needs. These findings informed our resulting Implementa-
ity, including failing to identify an IF. For large-scale implementation in tion Roadmap, which includes eleven critical implementation steps and
community-based clinics, augmenting REP with EF for sites that need evaluative metrics for investigators implementing CCIs to consider
additional support may be more feasible, and ultimately more effective, across pre-implementation, implementation, and sustainability phases.
than a more intensive EF/IF augmentation. Conclusions
Trial Registration: ClinicalTrials.gov NCT02151331 Maximal CCI public health impact requires improved reach. Our Imple-
mentation Roadmap provides a clear and practical guide for early stage
References community CCI implementation efforts, and ensure practitioners collect
1. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: key metrics and systematically address barriers in ways that are founda-
recommendations for specifying and reporting. Implement Sci. tional for larger scale, sustainable implementation efforts.
2013;8:139. doi:10.1186/1748-5908-8-139
2. Bonham AC, Solomon MZ. Moving comparative effectiveness research
References
into practice: implementation science and the role of academic
1. Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ,
medicine. Health Aff. 2010;29.10:1901-1905.
Aarons GA, Hoagwood KE, Evans AC, Hurford MO, Rubin R, Hadley T,
3. Atkins D, Kupersmith J. Implementation research: A critical component of
Mandell DS, Barg FK. A multi-level examination of stakeholder perspectives
realizing the benefits of comparative effectiveness research. Am J Med.
of implementation of evidence-based practices in a large urban publicly-
2010;123(12):e38-e45.
funded mental health system. Adm Policy Ment Health. 2016;43(6):893-908.
4. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-
2. Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, Padek M.
implementation hybrid designs: combining elements of clinical effective-
Sustainability of evidence-based healthcare: research agenda, methodological
ness and implementation research to enhance public health impact.
advances, and infrastructure support. Implement Sci. 2015;10(88):1-13.
Med Care. 2012; 50(3):217-26.
3. Bolboli AS, Reiche M. A model for sustainable business excellence:
5. Kilbourne, AM, Almirall, D, Eisenberg, D, Waxmonsky, J, Goodrich, DE,
implementation and the roadmap. TQM J. 2013;25(4), 331-346.
Fortney, JC, Kyle, J. Protocol: Adaptive Implementation of Effective
4. Heller D J, Hoffman C, Bindman AB. Supporting the needs of state health
Programs Trial (ADEPT): Cluster randomized SMART trial comparing a
policy makers through university partnerships. J Health Polit Policy Law.
standard versus enhanced implementation strategy to improve
2014;39(3):667-677.
outcomes of a mood disorders program. Implement Sci. 2010;9(1):132.
A10
A9 The collaborative chronic care model for mental health conditions:
A community-based implementation roadmap to inform from partnered implementation trial to scale-up and spread
scalability, sustainability, and spread of evidence-based Mark Bauer1,2, Kendra Weaver3, Bo Kim1,2, Christopher Miller1,2, Robert
collaborative care interventions Lew1,4, Kelly Stolzmann1, Jennifer Sullivan1,4, Rachel Riendeau1,5, Samantha
Amy Rusch, Shawna Smith, Lindsay Decamp, Celeste Liebrecht, Gregory Connolly1,2, Jeffery Pitcock6, Stig Ludvigsen1, A. Rani Elwy1,7
1
Dalack, Amy Kilbourne Veterans Health Administration, Boston, MA, USA; 2Department of
Institute for Healthcare Policy and Innovation University of Michigan, Psychiatry, Harvard Medical School Boston, MA, USA; 3VA Office of Mental
Ann Arbor, MI, USA Health and Suicide Prevention, Washington, DC, USA; 4School of Public
Correspondence: Amy Rusch ([email protected]) Health, Boston University; 5Department of Anthropology, University of
Implementation Science 2020, 15(Suppl 2):A9 Iowa, Iowa City, IA, USA; 6Central Arkansas Veterans Healthcare System,
Little Rock, AR, USA; 7Department of Psychiatry and Human Behavior,
Background Brown University Warren Alpert School of Medicine, Providence, RI, USA
Evidence-based collaborative care interventions (CCIs) can improve health Correspondence: Mark Bauer ([email protected])
by mitigating the access gap in mental health care treatment. However, Implementation Science 2020, 15(Suppl 2):A10
Background A11
Collaborative Chronic Care Models (CCMs) have extensive con- A randomized stepped wedge hybrid-II trial to implement the
trolled trial evidence for effectiveness in serious mental illnesses collaborative chronic care model in VA general mental health
[1-2], but there is little evidence regarding feasibility or impact clinics
in typical practice conditions. In partnership with the VA Office Christopher Miller1, Bo Kim1, Robert Lew1, Kelly Stolzmann1, Jennifer
of Mental Health and Suicide Prevention (OMHSP) we Sullivan1, Rachel Riendeau2, Jeffery Pitcock3, Alicia Williamson4, Samantha
conducted a randomized, stepped wedge implementation trial Connolly1, A. Rani Elwy5, Kendra Weaver6, Mark Bauer1
1
using blended internal-external facilitation [3] to implement VA Boston Healthcare System, Boston, MA, USA; 2Department of
CCMs in Behavioral Health Interdisciplinary Program (BHIP) Anthropology, University of Iowa, Iowa City, IA, USA; 3Central Arkansas
teams in the general mental health clinics of nine VA medical Veterans Healthcare System, Little Rock, AR, USA; 4School of Information,
centers [4-5]. Based on experience in this trial, OMHSP launched University of Michigan, Ann Arbor, MI, USA; 5Brown University Warren
an initiative to scale-up and spread the implementation effort Alpert School of Medicine, Providence, RI, USA; 6VA Office of Mental
more broadly. Health and Suicide Prevention, Washington, DC, USA
Materials and Methods Correspondence: Christopher Miller ([email protected])
Our research team and OMHSP engaged with Transformational Implementation Science 2020, 15(Suppl 2):A11
Coaches (T-Coaches) from the VA Office of Veterans Access to
Care to serve as external facilitators to engage additional VA Background
medical centers across the country. T-Coaches are senior facilita- Collaborative Chronic Care Models (CCMs) have extensive controlled
tors with skills in team-building and process redesign from trial evidence for effectiveness in serious mental illnesses [1], but
diverse professional disciplines. Trial external facilitators and there is little evidence regarding feasibility or impact in typical prac-
OMHSP leadership trained 17 T-Coaches in methods used in the tice conditions. We determined the effectiveness of implementation
trial. Sites were recruited by OMHSP. Blended facilitation was facilitation on establishing the CCM in mental health teams, and its
conducted for 12 months as in the implementation trial. Each impact on health outcomes of team-treated individuals.
of the T-Coaches partnered with a BHIP-CCM subject matter Materials and Methods
expert for the effort, and they conferred on a regular basis We used a randomized stepped wedge trial in Behavioral Health
throughout the year. Interdisciplinary Program (BHIP) teams in outpatient general mental
Results health clinics of nine VA facilities, using blended internal-external fa-
Thirty-nine sites were approached; of these 35 (89.7%) signed a cilitation. Facilitation combined a study-funded external facilitator
letter of agreement. Of these, 28 facilities (80.0%) completed a with a facility-funded internal facilitator working with a designated
site visit and entered the ongoing virtual facilitation process. Of team for one year. We hypothesized that facilitation would be associ-
these, 21 facilities (75.0%) completed the one-year facilitation and ated with improvements in both implementation and intervention
submitted CCM-concordance process summaries. The proportion outcomes (hybrid-II trial) [2]. Implementation outcomes included the
of CCM-concordant processes ranged widely across facilities, with clinician Team Development Measure (TDM) and proportion of CCM-
the more concordant sites equaling rates seen in the implemen- concordant team care processes. The study was powered for the pri-
tation trial and a broader low-end distribution (trial: 44-89, T- mary health outcome, VR-12 Mental Component Score (MCS). All Vet-
Coach scale-up: 13-93%). erans treated by designated teams were included for hospitalization
Conclusions analyses, based on administrative data; a randomly selected sample
In summary there was, not surprisingly, a broader range of was identified for health status interview. Individuals with dementia
CCM-concordance among these scale-up sites compared to the were excluded. For implementation outcomes, 62 clinicians were sur-
implementation trial. Nonetheless, taken together, the two veyed; site process summaries were rated for CCM concordance.
BHIP-CCM implementation efforts reached 30 VA medical Results
centers, of which 17 (56.7%) aligned over half of designated The population (n=5,596) included 881 (15%) women, average age
care processes with the evidence-based CCM. With strong oper- 52.2+14.5. The interviewed sample (n=1,050) was similar, but over-
ational partnerships and support, implementation trial efforts sampled for women (n=210, 20.0%). Facilitation was associated with
can be scaled up and spread to achieve broader healthcare improvements in TDM subscales for role clarity and team primacy.
system impact. Percentage of CCM-concordant processes achieved varied (44-89%).
Trial Registration: ClinicalTrials.gov NCT02543840 No improvement in veteran self-ratings, including the primary out-
come, was seen. However, in post-hoc analyses MCS improved in vet-
erans with >3 treated mental health diagnoses versus others. Mental
References
health hospitalization rate demonstrated a robust drop during facili-
1. Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer
tation; this finding withstood four internal validity tests [3].
MS. Comparative effectiveness of collaborative chronic care models for
Conclusions
mental health conditions across primary, specialty, and behavioral health
Working solely at the clinician level with minimal study-funded sup-
care settings: systematic review and meta-analysis. Am J Psychiatry. 2012;
port, CCM implementation yielded provider and Veteran benefits. Al-
169:790-804.
though impact on self-reported overall population health status was
2. Miller CJ, Grogan-Kaylor A, Perron BE, Kilbourne AM, Woltmann E,
negligible, health status improved for complex individuals, and
Bauer MS. Collaborative chronic care models for mental health con-
hospitalization rate declined. Facilitating CCM implementation pro-
ditions: cumulative meta-analysis and metaregression to guide fu-
vides a potential model for realigning VA outpatient general mental
ture research and implementation. Med Care. 2013;51:922-930.
health care with an evidence-based model that improves provider
3. Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney
team function and Veteran outcomes.
JC. Outcomes of a partnered facilitation strategy to implement
Trial Registration: ClinicalTrials.gov NCT02543840
primary care-mental health. J Gen Intern Med. 2014;29 Suppl 4:904-
912.
4. Bauer MS, Miller C, Kim B, Lew R, Weaver K, Coldwell C, Henderson K, References
Holmes S, Seibert MN, Stolzmann K, Elwy AR, Kirchner J. Partnering with 1. Miller CJ, Grogan-Kaylor A, Perron BE, Kilbourne AM, Woltmann E, Bauer
health system operations leadership to develop a controlled MS. Collaborative chronic care models for mental health conditions:
implementation trial. Implement Sci. 2016;11:22. cumulative meta-analysis and meta-regression to guide future research
5. Bauer MS, Miller C, Kim B, Lew R, Stolzman K, Sullivan J, Reindeau R, and implementation. Med Care. 2013;51:922-930.
Pitcock J, Williamson A, Connolly S, Elwy AR, Weaver K. Effectiveness of 2. Bauer MS, Miller CJ, Kim B, Lew R, Weaver K, Coldwell C, Henderson K,
implementing a Collaborative Chronic Care Model for clinician teams on Holmes SK, Nealon-Seibert M, Stolzmann K, Elwy AR, Kirchner J. Partner-
patient outcomes and health status in mental health: a randomized ing with health system operations leadership to develop a controlled im-
clinical trial. JAMA Netw Open. 2019;2:e190230. plementation trial. Implement Sci. 2016;11(22):1-11.
3. Bauer MS, Miller CJ, Kim B, Lew R, Stolzmann K, Sullivan J, Riendeau R, 2. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the
Pitcock J, Williamson A, Connolly S, Elwy AR, Weaver K. Effectiveness of Leadership and Organizational Change for Implementation (LOCI)
implementing a collaborative chronic care model for clinician teams on intervention in substance abuse treatment: A cluster randomized trial
patient outcomes and health status in mental health: a randomized study protocol. Implement Sci. 2017;12:29.
clinical trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/ 3. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across
jamanetworkopen.2019.0230. systems and organizations to develop a strategic climate for evidence-
based practice implementation. Annu Rev Public Health. 2014; 35:255-
274.
A12
Development, adaptation, and preliminary evaluation of the
Leadership and Organizational Change for Implementation A13
strategy Testing a multi-level implementation strategy for two evidence-
Mark G. Ehrhart1, Marisa Sklar2,3, Kristine Carandang2,3, Melissa R. Hatch2,3, based autism interventions
Joanna C. Moullin4, Gregory A. Aarons2,3 Lauren Brookman-Frazee1,2, Aubyn Stahmer3, Allison Jobin1,2, Kristine
1
Psychology Department, University of Central Florida, Orlando, FL, USA; Carandang1,2
2 1
Department of Psychiatry, University of California San Diego, San Diego, Department of Psychiatry, University of California San Diego, San Diego,
CA, USA; 3Child and Adolescent Services Research Center, University of CA, USA; 2Child and Adolescent Services Research Center, University of
California San Diego, San Diego, CA, USA; 4Faculty of Health Sciences, California San Diego, San Diego, CA, USA; 3MIND Institute, University of
Curtin University, Perth, Australia California Davis, Davis, CA, USA
Correspondence: Gregory A. Aarons ([email protected]) Correspondence: Lauren Brookman-Frazee ([email protected])
Implementation Science 2020, 15(Suppl 2):A12 Implementation Science 2020, 15(Suppl 2):A13
Background Background
This presentation describes the development of the Leadership and Children with ASD are a high priority population served in multiple
Organizational Change for Implementation (LOCI) strategy [1], subse- public service systems. Evidence-based behavioral interventions are
quent adaptation, and preliminary data for implementing Motivational available [1-2], however, they are not routinely delivered in commu-
Interviewing (MI) in substance abuse treatment settings [2]. LOCI is an nity care [3-6]. In response, our research groups used community-
implementation strategy developed to align higher level organizational partnered approaches to adapt and test ASD interventions for rou-
strategies with first-level leader development to create a strategic tine delivery - “AIM HI” in children’s mental health [7] and “CPRT” in
organizational climate to support implementation and sustainment of education [8]. We identified implementation leadership and climate
evidence-based practices (EBPs) [3]. Adaptations to the general design as key implementation mechanisms in recent community effective-
of LOCI that have occurred over time, as well as adaptations built into ness trials. We are conducting two, coordinated studies testing the
LOCI to tailor the strategy to particular settings will be described. LOCI effectiveness of an adapted version of the Leadership and
provides a general structure, curricula, and process for leading imple- Organizational Change for implementation (LOCI) strategy [9] as part
mentation, allowing for flexibility so that leaders across levels can of an implementation package [10]. This presentation describes the
prioritize issues most relevant at a given time for a given context. We application of LOCI in two ASD services contexts for two EBIs.
also describe mechanisms of change central to LOCI. Materials and Methods
Materials and Methods The TEAMS project includes two linked randomized Hybrid Type 3 im-
The current study involves 3 cohorts of 20 clinics in which clinics are be- plementation trials to test two implementation strategies when paired
ing randomized to LOCI vs. webinar control conditions. For this presenta- with AIM HI or CPRT and examine mechanisms of these strategies, in-
tion we utilize data from the first cohort 19 clinics and quantitatively cluding implementation leadership and climate. The TEAMS Leadership
examine differences in implementation leadership and climate by condi- Institute (TLI) applies LOCI as follows: (1) uses the LOCI components
tion. Qualitative data were collected from and analyzed to identify factors linked to mechanisms identified in the AIM HI and CPRT community ef-
influencing EBP implementation. Fourteen leaders across intervention fectiveness trials (i.e. implementation leadership and climate modules);
and control conditions responded to an online survey and provided rank- (2) targets executive and mid-level leaders required to coordinate im-
ings (order of importance) and ratings of factors affecting EBP implemen- plementation; and (3) targets implementation of specific ASD interven-
tation. These data were supplemented with qualitative interviews. tions. We present process data on TLI implementation and initial
Results themes from qualitative interviews to examine leader perceptions of
Preliminary results from the first cohort demonstrated that LOCI, com- the utility of TLI components and their impact on EBI implementation.
pared to the control condition, showed significant improvements in im- Results
plementation leadership and implementation climate. Qualitative To date, TLI has been conducted in 18 programs/districts in three
analyses showed that compared to control leaders, LOCI leaders were California counties including 18 workshops and 152 coaching calls.
more focused on staff competency and overcoming resistance. Time, Preliminary themes from interviews with 6 leaders who completed
turnover, and the influence of external contracts emerged as key themes. TLI indicate that TLI is feasible and useful to (1) convey to staff the
Conclusions importance of systematically planning EBI implementation, and (2) to
The LOCI implementation strategy was designed to improve general maintain leader motivation and focus on executing strategic initia-
and implementation leadership, subsequent implementation climate, tives around AIM HI and CPRT amidst competing demands.
and provider implementation behaviors including the adoption and Conclusions
use of EBP with fidelity. The ultimate goal is to improve client en- Preliminary data indicate the TEAMS application of LOCI is feasible
gagement in services and patient outcomes. Preliminary results sug- and perceived as effective in facilitating the implementation of two
gest that LOCI can improve the context for implementation or EBPs, ASD interventions. Future analyses will examine the impact of LOCI
and that LOCI can help to focus leaders’ attention on improving on targeted mechanisms – implementation leadership and climate.
implementation. Trial Registration: ClinicalTrials.gov NCT03380078
Trial Registration: ClinicalTrials.gov NCT03042832
References
References 1. National Autism Center. National Standards Project, Phase 2. 2015. http://
1. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and www.nationalautismcenter.org/national-standards-project/phase-2/.
organizational change for implementation (LOCI): a randomized mixed 2. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, Brock ME,
method pilot study of a leadership and organization development Plavnick JB, Fleury VP, Schultz TR. Evidence-based practices for children,
intervention for evidence-based practice implementation. Implement Sci. youth, and young adults with autism spectrum disorder: a comprehen-
2015;10:11 sive review. J Autism Dev Disord. 2015;45(7):1951–1966.
3. Brookman-Frazee L, Baker-Ericzén M, Stadnick N, Taylor R. Parent perspec- This study will provide knowledge about the effect of the LOCI pro-
tives on community mental health services for children with autism gram within a Norwegian context. As such, the results might inform
spectrum disorders. J Child Fam Stud. 2012;21(4):533–544. evidence-supported implementation strategies that could help sus-
4. Brookman-Frazee L, Drahota A, Stadnick N, Palinkas LA. Therapist pertain national-wide implementation of evidence-based trauma treat-
spectives on community mental health services for children with autism ment and increase the quality and effectiveness of Norwegian health
spectrum disorders. Adm Policy Ment Health. 2012;39(5):365-373. services.
5. Brookman-Frazee L, Taylor R, Garland AF. Characterizing community-based
mental health services for children with autism spectrum disorders and dis- References
ruptive behavior problems. J Autism Dev Disord. 2010; 40(10):1188–201. 1. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the
6. Stahmer AC, Collings NM, Palinkas LA. Early intervention practices for Leadership and Organizational Change for Implementation (LOCI)
children with autism: descriptions from community providers. Focus Intervention in substance abuse treatment: a cluster randomized trial
Autism Other Dev Disabil, 2005;20(2):66-79. study protocol. Implement Sci. 2017;12(1):29.
7. Brookman-Frazee L, Roesch S, Chelbowski C, Baker-Ericzen, Ganger W. Ef- 2. Egeland KM, Skar AMS, Endsjø M, Laukvik EH, Bækkelund H, Babaii A,
fectiveness of training therapists to deliver an individualized mental health Granly LB, Husebø GK, Borge RH, Ehrhart MG, Sklar M. Testing the
intervention for children with ASD in publicly funded mental health ser- leadership and organizational change for implementation (LOCI)
vices: a cluster randomized clinical trial. JAMA Psychiatry. 2019;76(6):574-583. intervention in Norwegian mental health clinics: a stepped-wedge cluster
8. Suhrheinrich J, Rieth SR, Dickson KS, Roesch S, Stahmer AC. Classroom randomized design study protocol. Implement Sci. 2019;14(1):28.
pivotal response teaching: Teacher training outcomes of a community 3. Aarons GA, Sklar M, Mustanski B, Benbow N, Brown CH. “Scaling-out”
efficacy trial. Teach Educ Spec Educ. 2019. https://doi.org/10.1177/ evidence-based interventions to new populations or new health care de-
0888406419850876 livery systems. Implement Sci. 2017;12(1):111.
9. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the
Leadership and Organizational Change for Implementation (LOCI)
Intervention in substance abuse treatment: a cluster randomized trial A15
study protocol. Implement Sci. 2017;12(1):29. Making sense of context: a systematic review
10. Brookman-Frazee L. Stahmer AC. Effectiveness of a multi-level implemen- Lisa Rogers, Aoife DeBrún, Eilish McAuliffe
tation strategy for ASD interventions: study protocol for two linked clus- School of Nursing, Midwifery and Health Systems, University College
ter randomized trials. Implement Sci. 2018;13(66). Dublin, Dublin, Ireland
Correspondence: Lisa Rogers ([email protected])
A14
Translation and adaptation of LOCI for implementation of Background
evidence-based treatment for PTSD in Norwegian child and adult The uptake of evidence-based healthcare interventions is challen-
mental health care services ging, with, on average, a 17-year time gap between the generation
Erlend Høen Laukvik, Ane-Marthe Solheim Skar, Karina M. Egeland of evidence and implementation of interventions into routine prac-
Norwegian Centre for Violence and Traumatic Stress Studies, Oslo, Norway tice [1]. Although contextual factors such as culture are strong influ-
Correspondence: Erlend Høen Laukvik ([email protected]) ences for successful implementation [2], context remains a poorly
Implementation Science 2020, 15(Suppl 2):A14 understood construct, with a lack of consensus regarding how it
should be defined and accounted for within research [3]. A system-
Background atic review was conducted to address this issue by providing an
The Norwegian Centre for Violence and Traumatic Stress Studies (NKVT insight into how context is defined and assessed within healthcare
S) is commissioned by the Ministry of Health and Care Services to im- implementation science literature and develops a definition to better
plement evidence-based treatment for post-traumatic stress disorder enable effective measurement of context.
(PTSD) in both child and adult mental health care services. As part of Materials and Methods
this effort, the Leadership and Organizational Change for Implementa- The databases of PubMed, PsycInfo, CINAHL, and EMBASE were
tion (LOCI) [1] was translated and adapted for the implementation of searched. English language empirical studies published in the previous
trauma treatment in Norwegian health trusts. The aim of the project is 10 years were included if context was treated as a key component in
to evaluate the effectiveness of LOCI in supporting the implementation implementing a healthcare initiative. Articles also needed to provide a
of evidence-based treatment for PTSD in Norwegian specialized mental definition and measure of context in order to be included. Results were
health clinics [2]. The presentation will identify the implementation de- synthesised using a narrative approach and supported using PRISMA
terminants, targets, and mechanisms being examined. guidelines for the conduct and reporting of systematic reviews.
Materials and Methods Results
The study is a Type III scale-out project [3]. Several a-priori adaptations The searches yielded 3,021 records of which 64 met the eligibility cri-
were made, including translation of the LOCI materials into Norwegian, teria and were included. Studies used a variety of definitions. Some
and tailoring of the LOCI fidelity tool. The study design is a stepped listed contextual factors (n=19) while others documented sub-
wedge cluster randomized trial with random and sequential enrollment elements of a framework that included context (n=19). Remaining ar-
of clinics into three cohorts, with crossover of clusters from control con- ticles provided a rich definition of an aspect of context (n=14) or
ditions to active intervention conditions based on time intervals. Execu- context generally (n=12). Quantitative studies mostly employed the
tives, clinic leaders, and therapists complete surveys assessing Alberta Context Tool while qualitative papers used a variety of frame-
leadership and implementation climate at baseline, 4, 8, 12, 16, and 20 works with Promoting Action on Research Implementation in Health
months. At baseline, all therapists at the participating clinics were Services framework the most highly cited. Mixed methods studies
trained in trauma screening and a sub sample in the treatment models used diverse approaches to assess context. Some used frameworks
for PTSD (TF-CBT, EMDR, CT-PTSD), and units were randomly assigned to inform the methods chosen while others used quantitative mea-
to one of three cohorts. In addition, the strategy uses the 360 degrees sures to inform qualitative data collection. Most papers (n=50)
assessments to inform subsequent work on tailored leadership and cli- applied the chosen measure to all aspects of study design with a ma-
mate development plans to enhance implementation. Therapy sessions jority analysing context at an individual or level (n=51).
are audio or video recorded and scored for fidelity. Patients complete Conclusions
surveys assessing symptom development during the therapy process. This review highlighted inconsistencies in defining and measuring
Results context which supported the development of an enhanced under-
Consistent with the LOCI theoretical model, assessment of mecha- standing for this construct. By providing this consensus, improve-
nisms will examine the effects of leadership on EBP fidelity through ments in implementation processes may result as greater
its effect on implementation climate. understanding will help researchers appropriately account for con-
Conclusions text in research.
References A17
1. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An DIY implementation: lessons from a practitioner-led
introduction to implementation science for the non-specialist. BMC Psy- implementation of an evidence-based practice
chol. 2015;3(1). doi:10.1186/s40359-015-0089-9 Sean Wright1, Sonia Combs2
1
2. Proctor EK, Powell BJ, Baumann AA, Hamilton AM, Santens RL. Writing Lutheran Community Services Northwest, Spokane, WA, USA; 2Cor
implementation research grant proposals: ten key ingredients. Counseling and Wellness, Spokane, WA, USA
Implement Sci. 2012;7(96). http://www.implementationscience.com/ Correspondence: Sean Wright ([email protected])
content/7/1/96. Accessed May 15, 2018. Implementation Science 2020, 15(Suppl 2):A17
3. Nilsen P. Making sense of implementation theories, models, and
frameworks. Implement Sci. 2015;10(1):53. doi:10.1186/s13012-015-0242-0 Background
Reports of implementation efforts initiated at the practitioner level
are uncommon. To address this gap, we describe the results of and
A16 lessons from an ongoing practitioner-led implementation of Accept-
Implementing mental health assessment in a juvenile detention ance and Commitment Therapy (ACT), an evidence-based practice, in
behavioral health unit: lessons learned from a community a community mental health center team.
academic partnership Materials and Methods
Brittany Rudd1, Jacquelyn George2, Lauren Cliggitt3, Sean We used a variety of implementation strategies (mostly training) during
Snyder4, Mynesha Whyte4, Rinad S. Beidas1 an ongoing implementation of ACT. Initially, we conducted a mixed
1
Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA; methods study of the facilitators and barriers to implementation, collect-
2
College of Public Health, Temple University, Philadelphia, PA, USA; ing qualitative and quantitative survey data anonymously at two time
3
Community Behavioral Health, Philadelphia, PA, USA; 4Hall Mercer Community points, sampled from all clinical staff (N=39) at our agency. The survey
Mental Health Center, University of Pennsylvania, Philadelphia, PA, USA measured attitudes, knowledge, experience, and acceptability of the EBP.
Correspondence: Brittany Rudd ([email protected]) We assessed the significance of changes in Likert ratings using the sign
Implementation Science 2020, 15(Suppl 2):A16 test [1]. We used thematic analysis to code qualitative data. Recently,
penetration was measured by relative use of ACT in a one-month sample
Background of progress notes and by the relative percentage of team members using
The dual objectives of juvenile justice are to assure youth safety while ACT. Implementation strategies used were identified by retrospective re-
in custody and to facilitate rehabilitation. Suicide is the second leading view and coded in accordance with the Expert Recommendations for
cause of death among 10-25 year olds [1], and is four times more likely Implementing Change (ERIC) project [2] and classified into concept map-
among youth who enter juvenile justice (JJ) settings [2]. As youth in ju- ping clusters for ERIC strategies [3]. We created a timeline of implementa-
venile detention are at risk for engaging in suicidal behaviors, it is crit- tion activities and identified key individuals who facilitated these activities.
ical that behavioral health clinicians in juvenile detention settings Results
conduct systematic evidence-based suicide risk, as well as general men- 15 pairs of pre-post survey measures indicated that initial training was as-
tal health, assessment. Recommendations for assessment in juvenile sociated with increases in identification as an ACT therapist (Z=-2.12,p=
detention exist [3], but there is little guidance regarding how to imple- 0.035), perceived ability to demonstrate ACT (Z=-3.00,p=0.002), and a
ment them among behavioral health clinicians. The current presenta- trend toward increased use of ACT (Z=-1.90,p=0.055). Qualitative analyses
tion will describe a community academic (CAP) partnership, and the were consistent with the existing literature on facilitators and barriers to
process that the partners underwent to implement a systemic protocol EBP adoption in community mental health. ACT use in a recent one-
for assessing youth in a juvenile detention behavioral health unit. month window was evidenced with 7.9% of progress notes documenting
Materials and Methods use of ACT (baseline before implementation: 0%) and 32% of eligible cli-
A CAP was developed and a quality improvement procedure was uti- nicians documenting ACT use in progress notes (initial baseline: 0%). Evi-
lized to develop and implement the assessment protocol. dence for use of 21 of the 73 ERIC implementation strategies was
Results documented. The strategies are distributed across all 9 concept mapping
The CAP team included the service clinicians (Snyder and Whyte), and clusters, with the Train and Educate Stakeholders cluster most repre-
clinical supervisor (Cliggitt) in a behavioral health unit housed in a sented (5 of 11 strategies). Three key individuals were identified.
large, juvenile detention center in an urban city in Pennsylvania, as well Conclusions
as researchers from the University of Pennsylvania (Rudd, George, and Practitioner-led implementation is feasible. Implementation strategies
Beidas). The development of the assessment protocol was an iterative can inform practitioner efforts.
process that occurred over eight months. The process started with a
comprehensive review of current workflow and workflow infrastructure,
including how youth were referred to the behavioral health unit and References
the information behavioral health unit staff had about youth prior to 1. Roberson PK, Shema SJ, Mundfrom DJ, Holmes TM. Analysis of paired
their intake. Iterative changes to workflow procedures were needed, in- Likert data: how to evaluate change and preference questions. Fam Med.
cluding developing infrastructure to support assessment (e.g., develop- 1995;27(10):671-675.
ing report templates) during the behavioral health intake appointment. 2. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
Finally, several assessment measures were piloted to determine fit. MM, Proctor EK, Kirchner JE. A refined complication of implementation
Conclusions strategies: results from the Expert Recommendations for Implementing
The creation of a CAP was key to developing and implementing a com- Change (ERIC) project. Implement Sci. 2015;10:21.
prehensive and feasible mental health and suicide assessment protocol. 3. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith
Lessons learned from the application of implementation science to the JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize
juvenile detention context from the joint perspectives of researcher relationships among implementation strategies and assess their feasibility
(Rudd) and clinician (Synder) stakeholder perspectives will be presented. and importance: results from the Expert Recommendations for
Implementing Change (ERIC) study. Implement Sci. 2015;10:109.
References
1. Heron MP. Deaths: Leading causes for 2016. National Vital statistics A18
reports. 2018;67(6). Implementation of an educator participatory program for
2. Wasserman GA, McReynolds LS. Suicide risk at juvenile justice intake. improving work environments on health and wellbeing: a mixed
Suicide Life Threat Behav. 2006;36(2):239-249. methods approach
3. Grisso T, Underwood LA. Screening and assessing mental health and Lisa Sanetti, Alexandra Pierce, Michele Femc-Bagwell, Alicia Dugan
substance use disorders among youth in the juvenile justice system. a Neag School of Education University of Connecticut, Storrs, CT, USA
resource guide for practitioners. US Department of Justice. 2004. https:// Correspondence: Lisa Sanetti ([email protected])
www.ncjrs.gov/pdffiles1 /ojjdp/204956.pdf. Implementation Science 2020, 15(Suppl 2):A18
A chronic and increasing challenge to employee wellness in schools is Many evidence-based practices (EBPs) rely on multiple dissemination
teacher stress [1]. Teachers are tied with nurses as having the highest rates supports to assist scaling to achieve population benefits. Intermedi-
of daily stress among occupations [2]. Chronic high levels of teacher stress ary organizations (IOs) are often key in leveraging critical functions in
are associated with (a) increased rates of physical and psychological health the overall support system to enhance diffusion strategies [1-3]. Des-
problems, including anxiety, depression, cardiovascular disease, and poor pite the prevalence of intermediaries as important accelerators of
sleep quality; (b) poor job performance, including absenteeism, negative evidence-based practices, little is known about which IO strategies
interactions, poor relationships with students, and poor classroom man- are most effective in ensuring implementation and scaling success or
agement, and (c) poor student outcomes, including low rates of academic how strategies link to existing IO capacities [3]. Further, there is a
achievement, lower levels of social adjustment, and increased rates of dearth of information regarding IO capacity assessments relative to
problem behavior [1,3]. Further, chronic teacher stress is the primary factor their capabilities to effectively diffuse EBP’s or to perform active im-
associated with the high rate of teachers leaving the profession for rea- plementation support.
sons other than retirement, which has nearly doubled over the past 25 Materials and Methods
years, constituting the primary cause of teacher shortages nationwide [4]. Working with a public-private partnership (state government agencies
A critical need to address teacher health and wellbeing exists, yet, on and private funders) to scale Triple P statewide in North and South Car-
average, only 31.4% of schools offer workplace health and wellness pro- olina, both states have selected IOs to enhance statewide Triple P im-
motion programs; most of these programs are top-down, one-size-fits-all plementation. To assess IO capacity for planning and delivering
approaches that are either ineffective or unsustainable [5]. implementation supports, a tool was needed to establish baseline cap-
Materials and methods acity and to guide support planning. The Intermediary Organization
The purpose of this mixed methods study was to implement the Capacity Assessment (IOCA) was developed as an IO capacity assess-
Healthy Workplace Participatory Program (HWPP), an evidence-based ment tool aligned with Mettrick et al.’s five observed functions.
approach that engages front-line employees (i.e., teachers) and su- Results
pervisors (e.g., administrators) in a collaborative, iterative design of Early capacity assessment data and collaborative qualitative usability
workplace health and wellness interventions [6]. feedback from partners indicate that the tool is demonstrating practical
Results utility toward capacity assessments and planning for ongoing support.
This participatory approach allows for (a) identification of health and well- The IOCA appears to align well with Mettrick et al.’s functional groups
ness issues most salient to employees; (b) development of a wider range of IO support activities. Early feedback suggests that use of the tool also
of interventions as employees are more aware of complex interactions be- aids in transferring knowledge of implementation science-informed
tween their work organization, workplace, and lifestyle; and (c) identifica- strategies to IO partners in functionally informative, practical ways.
tion of potential intervention barriers and facilitators; (d) increased buy-in Conclusions
to problem definition and intervention design; and (e) establishment of a The IOCA is demonstrating good alignment with known classes of IO
supportive organizational culture and processes for a self-correcting and support functions. It is also providing practical usability relative to
sustainable health and wellness promotion program. The HWPP has been understanding baseline IO capacity to deliver ongoing supports for
shown to effectively increased employee health and wellbeing in a wide scaling of EBPs. IOs that have experience with the tool report im-
range of worksites [6]; this is the first implementation effort in schools. proved understanding of implementation science-informed strategies
Conclusions and tools that can better guide them in their support activities.
Results of focus groups as well as formative and summative data re-
lated to implementation and intervention processes, strategies, and References
outcomes across EPIS phases in two 3rd-5th grade elementary 1. Franks RP, Bory CT. Who supports the successful implementation and
schools in the Northeast will be presented. sustainability of evidence-based practices? Defining and understanding
the roles of intermediary and purveyor organizations. New Dir Child Ado-
lesc Dev. 2015; 149:41-56. doi:10.1002/cad.20112
References
2. Mettrick J, Harburger DS, Kanary PJ., Lieman RB, Zabel,M. Building cross-
1. Flook L, Goldberg S, Pinger L, Bonus K, Davidson R. Mindfulness for
system implementation centers: a roadmap for state and local child serv-
teachers: a pilot study to assess effects on stress, burnout, and teaching
ing agencies in developing Centers of Excellence (COE). Baltimore, MD:
efficacy. Mind Brain Educ. 2013;7(3):182-195.
The Institute for Innovation & Implementation, University of Maryland.
2. Gallup. State of America’s Schools 2014. http://www.gallup.com/services/
2015. https://archive.hshsl.umaryland.edu/handle/10713/7379.
178769/state-america-schools-report.aspx . Accessed 22 July 2018.
3. Proctor E, Hooley C, Morse A, McCrary S, Kim H, Kohl PL. Intermediary/
3. Roeser RW, Skinner E, Beers J, Jennings PA. Mindfulness training and
purveyor organizations for evidence-based interventions in the US child
teachers’ professional development: an emerging area of research and
mental health: characteristics and implementation strategies. Implement
practice. Child Dev Perspect. 2012;6(2):167-173.
Sci. 2019;14(1):3. doi:10.1186/s13012-018-0845-3
4. Goldring R, Taie S, Riddles M. Teacher attrition and mobility: results from
the 2012–13 teacher follow-up survey (NCES 2014-077). 2014. http://nce-
s.ed.gov/pubsearch. Accessed 3 April 2016. A20
5. Naghieh A, Montgomery P, Bonell CP, Thompson M, Aber JL. A tailored implementation approach to improving PTSD care in
Organisational interventions for improving wellbeing and reducing work- military treatment facilities: integrating practice-based knowledge
related stress in teachers. Cochrane Database Syst Rev. 2015;4:CD010306. and implementation science
6. Robertson M, Henning R, Warren N, Dove-Steinkamp M, Tibirica L, Bizarro A, David Riggs1, Katherine Dondanville2, Elisa Borah3, Craig Rosen4
CPH-NEW Research Team. The intervention design and analysis scorecard: A 1
Center for Deployment Psychology, Uniformed Services University,
planning tool for participatory design of integrated health and safety inter- Bethesda, MD, USA; 2Department of Psychiatry and Behavioral Sciences,
ventions in the workplace. J Occup Environ Med. 2013;55:S86-S88. University of Texas Health Science Center at San Antonio, San Antonio,
TX, USA; 3Department of Psychiatry, University of Texas at Austin, Austin,
A19 TX, USA; 4National Center for PTSD, VA Palo Alto Health Care System,
Assessing intermediary organization capacity for active Palo Alto, CA, USA
implementation support: development and collaborative early Correspondence: David Riggs ([email protected])
usability appraisal of an intermediary organization capacity Implementation Science 2020, 15(Suppl 2):A20
assessment tool
Robin Jenkins, William Aldridge, Rebecca Roppolo Background
Frank Porter Graham Child Development Institute, University of North The panel will discuss integration of practical lessons learned from
Carolina at Chapel Hill, Chapel Hill, NC, USA clinicians and administrators with principles of implementation sci-
Correspondence: Robin Jenkins ([email protected]) ence to develop a program to increase use of evidence-based psy-
Implementation Science 2020, 15(Suppl 2):A19 chotherapy (EBP) for PTSD in military treatment facilities (MTFs).
Despite efforts to train military providers in EBPs, only a minority of ser- tools that are used by intermediary organizations charged with scaling
vice members receive them [1]. Implementation barriers likely vary evidence-based practices for a large state system of behavioral health.
across MTFs, which differ in size, resources, command structure, and Materials and Methods
implementation climate. Increased use of EBPs likely requires a tailored The Center for Practice Innovations (CPI), at Columbia Psychiatry and
approach that aligns implementation strategies to local conditions [2]. the New York State Psychiatric Institute, is an intermediary organization
Materials and Methods whose mission is to support the New York State Office of Mental Health
The Targeted Assessment and Context-Tailored Implementation of in the use of EBPs throughout community-based mental health agen-
Change Strategies (TACTICS) program combines needs assessment, a cies in New York State. CPI’s role includes: (a) public awareness, and
rubric for aligning implementation strategies to local barriers and fa- education; (b) scalable dissemination of training in EBPs; (c) implemen-
cilitators, and external facilitation to help clinics enact a collabora- tation support through learning collaboratives; (d) quality improve-
tively developed implementation plan. Through experience working ment; and (e) outcome evaluation. We will describe empirical
with MTFs, the Center for Deployment Psychology (intermediaries) approaches to the development, dissemination of scalable training and
identified common implementation barriers and potential context- implementation support for a range of initiatives at CPI.
specific strategies to address them. These were augmented with add- Results
itional relevant strategies from the Expert Recommendations for Grounded in the Consolidated Framework for Implementation Research
Implementing Change project [3] and input from experienced imple- [2], we will present on the CPI practice change model and how the CFIR
menters. Barriers and facilitators in the resulting TACTICS rubric were assists in planning for post-training implementation support and the
then mapped backed to domains of the Consolidated Framework for identification of barriers and facilitators to implementation. Using the
Implementation Research [4]. published taxonomy, Expert Recommendations for Implementing
After getting leadership approval and identifying a site champion, the Change [3], we will describe a range of implementation strategies (e.g.,
five-month TACTICS process involves conducting needs assessment in- instructional design methods, user-centered design, stakeholder en-
terviews with relevant staff and reviewing clinic data to identify barriers gagement) that inform the development of scalable online training and
and facilitators, using the TACTICS rubric to identify potential change identify targets for post-training implementation activities. Lastly, we
targets and strategies to address local conditions, and meeting with will provide examples of online training evaluation [4] and challenges
staff to develop the implementation plan. This is followed by weekly faced in reporting on the impact of implementation strategies [5-6]
coaching calls (external facilitation) to support the champion in enact- within a large system of behavioral healthcare.
ing changes to increase use of evidence-based psychotherapy. Conclusions
Results Although a balancing act, it is possible for intermediary organizations
TACTICS rubric development is completed and is being pilot tested to remain flexible, efficient, and rapid in response to the mission of
at one site. After this development phase, TACTICS will be tested in a real-world dissemination and implementation of EBPs and use
stepped-wedge randomized trial in eight military treatment facilities. empirically-driven distance and E-learning and implementation sci-
Conclusions ence approaches. There are opportunities for mutual learning, syn-
Development of the TACTICS program was informed by intermediar- ergy and collaboration to advance the field of implementation
ies’ practical knowledge from military clinicians, implementation ex- science for researchers and practitioners.
perience, and by implementation science frameworks. If successful,
TACTICS provides a barrier-to-solution tailoring framework informed
References
by implementation practitioners, researchers, and local staff.
1. Franks RP, Bory CT. Who supports the successful implementation and
sustainability of evidence-based practices? Defining and understanding
References
the roles of intermediary and purveyor organizations. New Dir Child Ado-
1. Hepner KA, Roth CP, Sloss EM, Paddock SM, Iyiewuare PO, Timmer MJ,
lesc Dev. 2015; 149:41-56. doi:10.1002/cad.20112
Pincus HA. Quality of care for PTSD and depression in the military health
2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC.
system: Final report. RAND Health Q. 2018;7(3):3.
Fostering implementation of health services research findings into
2. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK,
practice: a consolidated framework for advancing implementation
Mandell DS. Methods to improve the selection and tailoring of
science. Implement Sci. 2009; 4:50.
implementation strategies. J Behav Health Serv Res. 2017;44(2):177-194.
3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
doi:10.1007/s11414-015-9475-6.
MM, Proctor EK, Kirchner JE. A refined compilation of implementation
3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM,
strategies: results from the Expert Recommendations for Implementing
Proctor EK, Kirchner JE. A refined compilation of implementation strategies:
Change (ERIC) project. Implement Sci. 2015;10:21.
results from the Expert Recommendations for Implementing Change (ERIC)
4. Kirkpatrick DKJ, Kirkpatrick JD. Evaluating training programs: the four
project. Implement Sci. 2015;10:21. doi:10.1186/s13012-015-0209-1.
levels (3rd Edition). San Francisco, CA: Berrett-koehler Publishers; 2019.
4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery, JC.
5. Proctor EK, Powell BJ, McMillen JC. Implementation strategies:
recommendations for specifying and reporting. Implement Sci. 2013; 8:139.
6. Gold R, Bunce AE, Cohen DJ, Hollombe C, Nelson CA, Proctor EK, Pope
science. Implement Sci. 2009; 4:50. doi:10.1186/1748-5908-4-50.
JA, DeVoe JE. Reporting on the strategies needed to implement proven
interventions: an example from a “real-world” cross-setting implementa-
A21 tion study. Mayo Clin Proc. 2016;91(8):1074-83.
Rubber meets the road: how one intermediary organization uses
implementation science to inform training and implementation A22
supports for a large state system of behavioral health Utilization of train-the-trainer programs to support the
Sapana Patel1,2, Lisa Dixon1,2 sustainability of evidence-based trauma-informed interventions:
1 the perspectives of model developers, trainers, and intermediary
Department of Psychiatry, Columbia University, New York, NY, USA;
2 agencies within the National Child Traumatic Stress Network
The New York State Psychiatric Institute, New York, NY, USA
Correspondence: Sapana Patel ([email protected]) Shannon Chaplo1, George Ake1,2, Lisa Amaya-Jackson1,2, Byron J. Powell3,
Implementation Science 2020, 15(Suppl 2):A21 Ginny Sprang4
1
Department of Psychiatry and Behavioral Science, Duke University
Background Medical Center, Durham, NC, USA; 2National Center for Child Traumatic
At federal, state, and local levels, stakeholders are focused on develop- Stress, Washington, DC, USA; 3Brown School, Washington University in
ing, disseminating, and implementing evidence-based practices (EBPs). St. Louis, St. Louis, MO, USA; 4Department of Psychiatry, University of
Intermediary organizations are entities that help agencies or systems Kentucky, Lexington, KY, USA
develop, implement, and sustain evidence-based practices [1]. Little is Correspondence: Shannon Chaplo ([email protected])
known about how implementation science frameworks, strategies and Implementation Science 2020, 15(Suppl 2):A22
Background Materials and Methods

Train-the-Trainer programs (TTTs) refer to “a program or course where This project will engage in a participatory process with key stake-
individuals in a specific field receive training in a given subject and in- holders to design implementation strategies to increase universal de-
struction on how to train, monitor, and supervise other individuals in pression screening in primary care. In particular, we fill focus on
the approach [1].” TTTs are implementation strategies intended to in- designing a subset of implementation strategies—nudges, as they
crease the reach and sustainment of evidence-based interventions in are called in behavioral economics [1]—that alter the choice architec-
mental health agencies, and address other challenges such as therapist ture, or the way options are presented to optimize choices. First, we
attrition and developer succession planning [2-3]. Several trauma- began by conducting an innovation tournament, a crowdsourcing
informed interventions for children have TTTs; however, there is no technique [2], with physicians, nurses, medical assistants, behavioral
standardized protocol for developing or delivering TTTs. As the need to health clinicians, and front-desk stuff currently involved in adminis-
disseminate and sustain trauma-informed interventions grows, the tering the electronic depression screener, the two-item Patient
need to develop guidelines for TTTs becomes imperative. The objective Health Questionnaire (PHQ-2) at their practices. The tournament will
of this project is to better understand the state of TTTs for trauma- generate ideas on how to increase the implementation of the PHQ-2.
informed interventions utilized by members and consumers of the Na- Next, we will fine-tune the ideas generated from the innovation tour-
tional Child Traumatic Stress Network (NCTSN). nament with a team of stakeholders, behavioral economists, and im-
Materials and Method plementation scientists using behavioral economic theory [3]. The
Duke University study staff partnered with members of the NCTSN product of this project will be a toolkit of implementation strategies,
Implementation Advisory Committee to develop a survey exploring that are theoretically motivated and acceptable to a range of rele-
TTTs in the NCTSN. The survey was designed to gather the perspec- vant stakeholders, a subset of which will be later refined through a
tive of developers of treatments, practices, and curricula; profes- more rigorous piloting process.
sionals that become trainers through TTTs; and agency training Results
directors that serve as consumers of TTTs. Developers will answer a The innovation tournament closes mid-April, 2019. Ideas from the
series of questions about the development and implementation of tournament will be refined into a toolkit of implementation strategies
their TTT program. Trainers and agency directors will be asked about by September 2019.
their experience participating in a TTT program. All respondents will Conclusions
be asked about the components of their TTTs, barriers to using or Our study responds to the need for interdisciplinary, theoretically in-
developing TTTs, and facilitators of developing or using TTTs. formed, and participatory approaches to designing implementation
Results strategies. The results from our work will shed light on whether these
No results are currently available. The study has secured IRB approval approaches show promise.
and the survey will launch in April 2019. Results will be analyzed in
summer 2019. References
Conclusions 1. Thaler RH, Sunstein CR. Nudge: Improving decisions about health, wealth,
Surveying each of these audiences will help us to better understand and happiness. Revised and Expanded Edition. New York, NY: Penguin
the varying components of TTTs, and barriers and facilitators of their Books; 2009.
use within the NCTSN. We plan to use the survey results for training 2. Terwiesch C, Mehta SJ, Volpp KG. Innovating in health delivery: the Penn
purposes and resource development to enhance the use of TTTs medicine innovation tournament. Healthc. 2013;1(1-2):37–41.
within the NCTSN (and in other relevant settings) to implement and 3. Eldredge LKB, Markham CM, Ruiter RA, Kok G, Fernandez ME, Parcel GS.
sustain trauma-informed interventions. Planning health promotion programs: an intervention mapping
approach. San Francisco: John Wiley & Sons; 2016.
References
1. Pearce J, Mann, MK, Jones, C, van Buschbach, S, Olff, M, Bisson, JI. The
most effective way of delivering a Train‐the‐Trainers program: a A24
systematic review. J Contin Educ Health Prof. 2012; 32:215-226. Using stakeholder values to promote implementation of an
2. Bero L, Grilli R, Grimshaw J, Harvey E, Oxman A, Thomson M. Closing the evidence-based mHealth intervention for addiction treatment in
gap between research and practice: an overview of systematic reviews of primary care
interventions to promote implementation of research findings by health Correspondence: Andrew Quanbeck ([email protected])
care professionals. BMJ. 1998;317:465–468. College of Engineering, University of Wisconsin – Madison, Madison, WI,
3. Yarber L, Brownson CA, Jacob RR, Baker EA, Jones E, Baumann C, USA
Deshpande AD, Gillespie KN, Scharff DP, Brownson RC. Evaluating a train- Implementation Science 2020, 15(Suppl 2):A24
the-trainer approach for improving capacity for evidence-based decision
making in public health. BMC Health Serv Res. 2015;15:547. Background
The majority of evidence-based practices do not find their way into
clinical use, including mobile health (mHealth) technologies. This
A23 presentation describes a novel decision-framing model that gives im-
Nudge yourself: stakeholder design of implementation strategies plementers a method for eliciting the perceived gains and losses that
that leverage insights from behavioral economics different stakeholder groups trade off when faced with the decision
Briana S. Last, Courtney Benjamin Wolk, Rinad S. Beidas of whether to adopt an evidence-based mHealth intervention.
Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA Materials and Methods
Correspondence: Briana S. Last ([email protected]) The decision-framing model integrates insights from behavioral eco-
Implementation Science 2020, 15(Suppl 2):A23 nomics [1,2] and game theory [3]. The approach was applied retro-
spectively in a parent implementation research trial that introduced
Background an mHealth system to 268 patients in three U.S. clinics offering pri-
Though several evidence-based practices (EBPs) exist for depression, mary and behavioral healthcare services. The mHealth system, called
only a fraction of individuals receive treatment. One major challenge to Seva, supports patients with addiction. Individual and group inter-
treatment is the identification of individuals in need of care. In response views were conducted to elicit stakeholder considerations from 23
to this need, health care systems such as the University of Pennsylvania clinic staff members and 6 patients who were involved in implement-
(Penn) have mandated universal screening of depression in primary ing Seva. Considerations were used to construct “decision frames”
care settings based on evidence. However, adherence to the mandate that trade off the perceived value of adopting Seva vs. maintaining
at Penn is much lower than anticipated (only 40% of eligible patients the status quo from each stakeholder group’s perspective. The face
are screened). In our study, we partnered with front line clinicians and validity of the decision-framing model was assessed by solicit-
staff to increase depression screening at Penn using innovative ap- ing feedback from the stakeholders whose input was used to
proaches from implementation science and behavioral economics. build it.
Results levels of engagement. However, we also identified barriers (e.g., ensur-

Primary implementation considerations were identified for each ing that the stakeholder voice was adequately represented throughout
stakeholder group. Clinic managers perceived the greatest potential all stages). The system-wide survey will be launched in March, 2019;
gain to be providing better care for patients, and the greatest poten- and will close April, 2019.
tial loss to be cost, expressed in terms of staff time, sustainability, Conclusions
and opportunity cost. All clinical staff considered time their foremost The approach that we took in designing implementation strategies is
consideration—primarily in negative terms (e.g., cognitive burden as- promising. Research is needed to test whether strategies developed
sociated with learning a new system) but potentially positively (e.g., via these methods are more effective than strategies developed
if Seva could automate functions done manually). Patients consid- through competing approaches.
ered safety (anonymity, privacy, and coming from a trusted source)
to be paramount. When considerations were compiled into decision A26
frames that traded off the gains and losses associated with adopting Leveraging normative pressure to increase data collection among
Seva, only one stakeholder group—patients—expressed a positive therapists working with children with autism
overall value, and these were the stakeholders who used Seva most. David S. Mandell, Heather Nuske, Emily Becker-Haimes
Conclusions Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA
This paper presents a systematic method of inquiry to elicit stake- Correspondence: Emily Becker-Haimes ([email protected])
holders’ considerations when deciding to adopt a new technology. Implementation Science 2020, 15(Suppl 2):A26
Stakeholder considerations may be used to adapt mHealth interven-
tions and tailor implementation, potentially increasing the likelihood of Background
implementation success for evidence-based practices and technologies. Evidence-based practices for children with autism generally follow
the principles of applied behavior analysis, which require frequent,
References systematic data collection. In Philadelphia, as in many systems, chil-
1. Tversky A, Kahneman D. The framing of decisions and the psychology of dren with autism often are accompanied by a one-to-one aide, who
choice. Science 1981; 211(4481):453-8. is responsible for collecting these data as part of implementing a
2. Kahneman treatment plan. Direct observations and interviews with these aides
D, Lovallo D, Sibony O. Before you make that big decision… Harv Bus Rev and their supervisors confirm that aides rarely collect data in a rigor-
2011;89(6):50-60. ous manner. These aides often work in isolation and rarely receive
3. Myerson consistent supervision, which may lead to the perception that data
RB. Game theory: analysis of conflict. Cambridge: Harvard University Press; collection is not expected of them, nor do people in their position
1991. collect data in this manner. We use participatory methods combined
with innovative methods borrowed from industry that incorporate
A25 the principles of behavioral economics to design implementation
Applying insights from participatory design to design strategies to increase aides’ data collection.
implementation strategies Materials and Methods
Rinad S. Beidas1, Nathaniel Williams2, Rebecca Stewart1 We partnered with five community agencies and used time-and-
1
Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, motion study methods, an observational technique drawn from scien-
USA; 2School of Social Work, Boise State University, Boise, ID, USA tific management, to understand how one-to-one aides collect data.
Correspondence: Rinad S. Beidas ([email protected]) This process involved querying aides about the decisions they made re-
Implementation Science 2020, 15(Suppl 2):A25 garding data collection in the moment. We used participatory design
strategies, including an innovation tournament—a method to crowd-
Background source strategy ideas from stakeholders—and a rapid-cycle approa-
Public behavioral health systems have increasingly invested in the imple- ch—a method that involves iterative testing and refining
mentation of evidence-based practices (EBPs), including Philadelphia’s implementation strategies—to increase one-to-one aides’ data collec-
Department of Behavioral Health. Training and technical assistance con- tion. We applied theoretical principles from behavioral economics to re-
tinue to be the most commonly used strategies to increase use of EBPs, fine the implementation strategies generated from the innovation
despite findings that organizational barriers matter. Few organizational tournament and to test them using our rapid cycling approach.
implementation strategies exist and little is known about how to best de- Results
sign organizational strategies to increase implementation of EBPs using Data collection is ongoing. Here we present on the time-and-motion
participatory design approaches. We partnered with front line clinicians studies and results of our innovation tournament. We provide a
to develop organizational implementation strategies to improve EBP im- framework for the rapid-cycle process that is ongoing at the time of
plementation in community mental health clinics. this presentation.
Materials and Methods Conclusions
We engaged in a three-step process to design organizational imple- This method of data collection in the service of identifying imple-
mentation strategies. First, we launched an innovation tournament to mentation strategies and rapidly testing them holds promise.
engage clinicians employed within the Philadelphia public behavioral
health system to crowd-source how their organizations can support A27
them to use EBPs. We held a community-facing event during which Applications of standardized patient methodology to measure
the 6 clinicians who submitted winning ideas presented their ideas fidelity in an implementation trial of the teen marijuana check-up
to 85 attendees representing a range of stakeholders. Second, we Bryan Hartzler, Denise Walker, Aaron Lyon, Kevin King, Lauren
worked with behavioral scientists to refine the ideas to optimize their Matthews, Tara Ogilvie, Devon Bushnell, Katie Wicklander
effectiveness. Third, we launched a system-wide survey targeting ap- University of Washington, Seattle, WA, USA
proximately 300 stakeholders to elicit preferences for the clinician Correspondence: Bryan Hartzler ([email protected])
generated organizational implementation strategies. Implementation Science 2020, 15(Suppl 2):A27
Results
We report on the outcomes of the innovation tournament and system- Background
wide survey. A total of 65 ideas were submitted in the innovation tour- A cornerstone of medical education, standardized patients (SPs) are
nament by 55 participants representing 38 organizations. The most increasingly incorporated in implementation trials for behavior ther-
common categories of ideas pertained to training (42%), compensation apies as a highly valid, advantageous approach to fidelity measure-
(26%), clinician support tools (22%), and EBP-focused supervision (17%). ment [1]. Such methodological benefits extend to SP involvement in
Using an innovation tournament to generate ideas for implementation behavioral rehearsal activities often included to support therapy
strategies was feasible and acceptable as demonstrated by the high training processes [2]. An ongoing implementation trial examining
the Teen Marijuana Check-Up (TMCU) [3], a school-based adaptation speed the translation of such interventions by accomplishing the
of motivational enhancement therapy, incorporates SPs for both pur- dual tasks of (a) establishing effectiveness of interventions as they
poses [4]. are being rolled out under real-world conditions and (b) identifying
Materials and Methods determinants of implementation that can assist with planning of fu-
In this trial, a set of SPs portray marijuana-using adolescent charac- ture scaling activities [2-3]. The current study aims to accomplish just
ters in dyadic interactions with participating school-based staff. As such a task through the replication and testing of Project Planned
components of TMCU training, two SP-involved training cases—each Outreach, Intervention, Naloxone, and Treatment (POINT), an ED-
offering consequence-free opportunities for staff to receive based peer support intervention aimed at connecting people with
performance-based trainer feedback—supplemented an initial work- opioid use disorder to medication assisted treatment (i.e., metha-
shop. As components of pre- and post-training outcome assess- done-, buprenorphine-, or naltrexone-facilitated treatment). This
ments, two more SP interactions provided behavioral outcome study is funded by a unique federal mechanism that aims to improve
measures. All four SP interactions involved travel to staff workplaces rapid translation of research to practice through academic-state part-
to record a simulated TMCU session, later scored for the following fi- nerships. In this presentation, we will provide an overview of our
delity indices: ratio of reflective listening statements to questions pragmatic hybrid design before focusing on results of the study’s 6-
(R:Q), percentage of ‘open-ended’ questions (%OQ), and percentage month pilot phase.
of ‘complex’ reflective listening statements (%CR). Materials and Methods
Results The researchers partnered with the Indiana Division of Mental Health
Recruited from seven high schools, twenty staff completed all four and Addiction to carry out this study. Per the project’s funding mech-
SP interactions. Pre-training SP interactions revealed variable staff anism, success of the pilot was to be determined by the achievement
performances, with two staff members achieving a TMCU proficiency of 3 milestones, including our ability to successfully replicate the
standard by exceeding benchmarks for all three fidelity indices. In POINT intervention with 75% fidelity to previously identified critical
SP-involved training cases, this proficiency standard was achieved by components within a new implementation context.
eight staff in an initial case, and by six staff in a more challenging lat- Results
ter case. In eventual post-training SP interactions, five staff met the Overall implementation of the study protocols was successful, with
TMCU proficiency standard. As for mean training impact, Cohen’s d only minor refinements to proposed procedures being required in
effect sizes suggest small-to-medium effects on R:Q (d=.20), %CR light of challenges with (a) data access, (b) recruitment, and (c) iden-
(d=.36), and %OQ (d=.43), and documented expected needs for sub- tification of the expansion hospitals. All three milestones were
sequent support of TMCU implementation via purveyor coaching as reached, including 77% fidelity to the original POINT programs’ com-
a targeted form of post-training technical assistance. ponents. Challenges in implementing protocols and reaching mile-
Conclusions stones resulted in refinements that improved the study design
This trial—wherein SP methodology further extends to monitoring of overall. The subsequent trial will add to the limited but growing evi-
TMCU fidelity in biannual assessments for two years after trainin- dence on ED-based peer supports.
g—includes SP roles in outcome assessment and training. Fidelity Conclusions
data from the collective SP interactions evidence sensitivity to hy- Capitalizing Indiana’s current efforts in order to study an already scal-
pothesized changes in staff learning, further supporting the use of ing and promising intervention is likely to lead to faster and more
SPs as means to measure and monitor fidelity in trials examining be- sustainable results with greater generalizability than traditional,
havior therapy implementation. efficacy-focused clinical research.
Trial Registration: ClinicalTrials.gov NCT03111667 Trial Registration: ClinicalTrials.gov NCT03336268
References References
1. Imel ZE, Baldwin SA, Baer JS, Hartzler B, Dunn C, Rosengren DB, Atkins 1. Volkow ND, Collins FS. The role of science in addressing the opioid crisis.
DC. Evaluating therapist adherence in motivational interviewing by N Engl J Med. 2017;377(4):391-394.
comparing performance with standardized and real patients. J Consult 2. Westfall JM, Mold J, Fagnan L. Practice-Based Research—“Blue Highways”
Clin Psychol. 2014; 82(3):472-481. on the NIH Roadmap. JAMA. 2007; 297(4):403-406.
2. Beidas RS, Cross W, Dorsey S. Show me, don’t tell me: behavioral 3. Bernet AC, Willens DE, Bauer MS. Effectiveness-implementation hybrid de-
rehearsal as a training and analogue fidelity tool. Cogn Behav Pract. signs: implications for quality improvement science. Implement Sci.
2014;21(1):1-11. 2013;8(Suppl 1):S2.
3. Walker DD, Stephens RS, Roffman R, Towe S, DeMarce J, Lozano B, Berg
B. Randomized controlled trial of motivational enhancement therapy
with nontreatment-seeking adolescent cannabis users: a further test of A29
the teen marijuana check-up. Psych Addict Behav. 2011;25(3):474-484. Evaluating associations between implementation barriers,
4. Hartzler B, Lyon AR, Walker DD, Matthews L, King KM, McCollister KE. strategies, and program performance: data from 140 substance
Implementing the teen marijuana check-up in schools—a study protocol. abuse treatment programs in an integrated healthcare system
Implement Sci. 2017;12(103). Eric Hermes1, Ilse Wiechers1,2
1
Department of Psychiatry, Yale University, New Haven, CT, USA;
2
Department of Veterans Affairs Office of Mental Health and Suicide
A28 Prevention, Washington, DC, USA
A hybrid type 1 design to facilitate rapid testing and translation of Correspondence: Eric Hermes ([email protected])
an emergency department-based opioid use disorder intervention Implementation Science 2020, 15(Suppl 2):A29
through an academic-state government partnership
Dennis Watson1, Alan McGuire2,3, Rebecca Buhner4, Krista Brucker5 Background
1
College of Medicine, University of Illinois at Chicago, Chicago, IL, USA; Associations between contextual barriers, implementation strategies,
2
Richard L. Roudebush VAMC, Indianapolis, IN, USA; 3Department of and program performance can be evaluated using data from on-
Psychology, Indiana University, Indianapolis, IN, USA; 4Indiana Division of going quality improvement programs in healthcare operations [1].
Mental Health and Addiction, Indianapolis, IN, USA; 5Emergency The Psychotropic Drug Safety Initiative (PDSI) is a system-wide pro-
Medicine, Indiana University School of Medicine, Indianapolis, IN, USA gram guiding quality improvement for psychotropic prescribing in
Correspondence: Dennis Watson ([email protected]) 140 Veterans Health Administration (VHA) facilities. In 2017, PDSI
Implementation Science 2020, 15(Suppl 2):A28 began a program to increase medication assisted therapy (MAT) for
Opiate Use Disorder (OUD) and Alcohol Use Disorder (AUD). This ana-
Background lysis characterizes perceived barriers, providing a foundation for ana-
The gravity of the opioid epidemic requires innovative and promising lyzing associations between barriers, implementation strategies, and
solutions that can be rapidly scaled [1]. Hybrid type 1 designs can program performance.
Materials and Methods Materials and Methods

Among six core policies, PDSI provides metrics of local MAT use and re- Preliminary data from 29 individual interviews and 15 focus groups with
quires facilities to identify a champion. Facility assessments are submit- 10 executives, 23 supervisors, and 80 providers across five agencies were
ted, which identify disorder focus (AUD and/or OUD) and perceived examined. Notes from coaching calls conducted with supervisors ran-
barriers (including 18 barriers previously identified by key informants as domized to the LOCI condition were explored to further contextualize en-
well as free text) [2]. Barriers are characterized by Consolidated Frame- gagement in the implementation strategy and MI implementation over
work for Implementation Research (CFIR) construct, selection frequen- time. The Framework Method [2] was used to synthesize data within-
cies, intercorrelations, and associations with facility characteristics [3]. and between-agencies and identify emergent themes in accordance with
Results the Exploration, Preparation, Implementation, Sustainment (EPIS) frame-
All 140 VHA facilities responded: 74 (52.9%) focused on AUD, 47 work [3] and Edgar Schein’s organizational culture change model [4].
(33.6%) on OUD, and 19 (13.6%) on both. Frequently selected bar- Results
riers, including free text, clustered in the “individual characteristic” Determinants of LOCI engagement and MI implementation included
and “inner setting” CFIR domains: “Patients frequently refuse treat- the structural make-up of the agency and/or clinic, timing of state-wide
ment or referral” (107 [76.4%]) and “Providers have too many com- policy initiatives, and professional role at the agency. Agency execu-
peting demands” (98 [70.0%]). Neither disorder focus nor frequency tives, supervisors, and providers all agreed that inadequate staffing and
of barrier identification varied significantly with level of MAT use at high turnover limited the time available for LOCI engagement and MI
the facility. There was moderate intercorrelation of selected barriers implementation. Participants detailed how the lack of basic resources,
(Cronbach’s alpha = 0.67). The barriers of “Not enough x-waiver pro- such as not having therapy rooms, negatively impacted their ability to
viders” and “MAT required enrollment in intensive treatment pro- participate in LOCI and implement MI. Additionally, all agreed that
grams” were selected more frequently in OUD-focused programs. changes in policy introduced new requirements (e.g., new EHR, billing
Associations with specific improvement strategies used and program and reporting requirements) that interfered with LOCI participation.
performance will be reported in June 2019. Conclusions
Conclusions In order for organizations to engage effectively in implementation strat-
The most frequently perceived barriers to increasing MAT were char- egies like LOCI, a foundation of basic supports and resources is needed.
acteristics of patients, providers, and the local organization. Associa- The introduction of such strategic initiatives are held to be a secondary
tions between barriers, disorder focus, and MAT use were weak. Data priority after basic organizational needs such as fiscal and operational via-
suggest a disconnect between perceived barriers and knowledge of bility are met. Therefore, understanding the determinants for establishing
organization performance. Linking program barriers, implementation a foundation for successful EBP implementation in real-world practice is
strategies, and performance may be an implicit assumption of necessary to ensure that implementation strategies are successful and
healthcare improvement programs. Analyzing these links demon- implementation outcomes are achieved. Implications for improving im-
strates intersections between implementation science research, qual- plementation strategies to target these determinants are discussed.
ity improvement practice, and health system policy. Trial Registration: ClinicalTrials.gov NCT03042832
1. Rogal SS, Yakovchenko V, Waltz TJ, Powell BJ, Kirchner JE, Proctor EK, 1. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the
Gonzalez R, Park A, Ross D, Morgan TR, Chartier M, Chinman MJ. The Leadership and Organizational Change for Implementation (LOCI)
association between implementation strategy use and the uptake of Intervention in substance abuse treatment: a cluster randomized trial
hepatitis C treatment in a national sample. Implement Sci. 2017;12(1):60 study protocol. Implement Sci. 2017;12(1):29.
2. Harris AHS, Ellerbe L, Reeder RN, Bowe T, Gordon AJ, Hagedorn H, Oliva E, 2. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the
Lembke A, Kivlahan D, Trafton JA. Pharmacotherapy for alcohol dependence: framework method for the analysis of qualitative data in multi-
perceived treatment barriers and action strategies among Veterans Health disciplinary health research. BMC Med Res Methodol. 2013;13:117.
Administration service providers. Psychol Serv. 2013;10(4):420–7 3. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic
3. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. review of the Exploration, Preparation, Implementation, Sustainment
Fostering implementation of health services research findings into (EPIS) framework. Implement Sci. 2019;14(1):1.
practice: a consolidated framework for advancing implementation 4. Schein E. Organizational culture and leadership. 5 ed. Hoboken, NJ: John
science. Implement Sci. 2009; 4(1):50. doi:10.1186/1748-5908-4-50 Wiley and Sons; 2017.
A30 A31
Setting the foundation for successful engagement with Involving patients, practitioners, and policy makers to develop a
implementation strategies: multilevel perspectives from substance theory-based implementation intervention to increase the uptake
use treatment agencies of diabetic retinopathy screening
Chariz Seijo1, Kendal Reeder1, Kristine Carandang1, Marisa Sklar1, Mark Fiona Riordan1, Emmy Racine1, Susan Smith2, Aileen Murphy1, John
Ehrhart2, Cathleen Willging3, Gregory Aarons1 Browne1, Patricia Kearney1, Sheena McHugh1
1 1
Department of Psychiatry, University of California San Diego, San Diego, School of Public Health, University College Cork, Cork, Ireland; 2Department
CA, USA; 2Department of Psychology, University of Central Florida, of General Practice, Royal College of Physicians in Ireland, Dublin, Ireland
Orlando, FL, USA; 3Pacific Institute for Research and Evaluation, Correspondence: Fiona Riordan ([email protected])
Calverton, MD, USA Implementation Science 2020, 15(Suppl 2):A31
Correspondence: Chariz Seijo ([email protected])
Implementation Science 2020, 15(Suppl 2):A30 Background
Despite evidence that diabetic retinopathy screening (DRS) is effective [1],
Background uptake remains sub-optimal in many countries, including Ireland [2-5]. Im-
A foundation of basic supports and resources is required for the plementation strategies to enhance uptake of interventions like DRS, are
successful implementation of evidence-based practices (EBPs). not always a good fit for the context in which they are used, or do not
Inner and outer context determinants for this foundation remain align with stakeholder preferences [6]. We report a systematic process
unclear. As part of a cluster randomized trial testing the Leader- combining theory, stakeholder consultation and existing evidence to de-
ship and Organizational Change for Implementation (LOCI) inter- velop an implementation intervention to increase DRS uptake.
vention for motivational interviewing (MI) implementation across Materials and Methods
substance use treatment agencies [1], we use qualitative methods Target behaviours were identified through stakeholder interviews (n=
to explore experiences of agency executives, supervisors, and 19), and an audit of screening attendance in two primary care centres.
providers regarding the multilevel determinants of strategic Using patient (n=48) and health care professional (HCP) (n=30) inter-
implementation. views, barriers and enablers were coded using the Theoretical Domains
Framework and mapped to behaviour change techniques. The APEASE Materials and Methods
(affordability, practicability, effectiveness, acceptability, side effects, In this mixed methods, pragmatic trial, 29 CHCs with a shared EHR were
equity) criteria were used to select intervention components. Effective- randomized to 3 Arms that received implementation support: 1) Imple-
ness of components and delivery modes was determined through a mentation Toolkit (CVD Bundle use instructions; Quality Improvement
rapid evidence review. Feasibility, local relevance and acceptability practice change techniques); 2) Toolkit + in-person training with follow-
were determined through a collaborative process; consensus meetings up webinars; or, 3) Toolkit, training, webinars, + offered practice facilita-
with patients (n=15) and HCPs (n=16), and key stakeholder consultation. All study CHCs also identified a Champion to oversee related clinic
tions, including the national DRS programme. activities. Statin prescription rates were compared across Arms, and with
Results those in >300 additional CHCs which received no implementation sup-
Three target behaviours were identified; patient registration by HCP, port, a non-randomized comparison group. Prescribing (per national
patient consent for the programme to hold their details, and patient at- guidelines) was measured from 12 months pre-intervention through 36
tendance. Patient barriers included confusion between screening and months post-intervention. We gathered qualitative data from the ran-
routine eye checks, forgetting, and fear of a negative result. Enablers in- domized CHCs via on-site observations, interviews, and phone calls.
cluded a recommendation from friends/family or HCPs, recognising Results
screening importance, and ownership over their condition. HCP barriers Statin prescribing increased pre- to post-intervention for all Arms; only
included the time to register patients impeded or supported by prac- Arm 2 demonstrated a statistically significant change relative to compari-
tice resources, and a lack of readily available information on uptake in son CHCs. Prescribing rates improved more in the study CHCs (7%, 8%,
their local area/practice. Consensus meeting participants agreed HCP- and 5% for Arms 1, 2 and 3 respectively) than the comparison CHCs (3%).
endorsed reminders and patient information leaflets were acceptable. These differences were not additive – CHCs that received more intensive
They felt certain delivery modes (i.e. in-person, phone and letter) were implementation support did not have greater improvements in prescrib-
feasible and equitable, while others may exclude some practices and ing rates. Qualitative data suggest numerous clinic- and intervention-level
patients (e.g., text messages). The final intervention comprises reim- factors underlying these results. Implementation strategies were not al-
bursement, training, audit/feedback and electronic prompts for HCPs, ways applied as planned: the Toolkit was infrequently used, webinar at-
and HCP-endorsed patient reminders with an information leaflet. tendance was poor, staff turnover was substantial, and few Arm 3 clinics
Conclusions were able to fully benefit from the offered practice facilitation.
A collaborative process involving multiple stakeholder consultations Conclusions
helped shape an intervention deemed acceptable to both patients This is one of the first studies to directly compare implementation
and HCPs. The feasibility of intervention delivery in real world pri- strategies. The strategies employed here were associated with small
mary care will be evaluated through a pilot trial. improvements in the study CHCs’ guideline-concordant prescribing.
Level of implementation support was less impactful than clinic ability
References to make changes. Guideline dissemination efforts should evaluate
1. Cheung N, Mitchell P, Wong TY. Diabetic retinopathy. Lancet. adopters’ needs / preferences so that subsequently deployed imple-
2010;376(9735):124-136. mentation strategies are well-received.
2. Zwarenstein M, Shiller SK, Croxford R, Grimshaw JM, Kelsall D, Paterson Trial Registration ClinicalTrials.gov NCT03001713
JM, Laupacis A, Austin PC, Tu K, Yun L, Hux JE. Printed educational
messages aimed at family practitioners fail to increase retinal screening References
among their patients with diabetes: a pragmatic cluster randomized 1. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, Glass
controlled trial [ISRCTN72772651]. Implement Sci. 2014;9(1):87. JE, York JL. A compilation of strategies for implementing clinical innovations
3. Millett C, Dodhia H. Diabetes retinopathy screening: audit of equity in in health and mental health. Med Care Res Rev. 2012;69(2):123-157.
participation and selected outcomes in South East London. J Med 2. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations
Screen. 2006;13(3):152-155. for specifying and reporting. Implement Sci. 2013;8:139.
4. Paz SH, Varma R, Klein R, Wu J, Azen SP. Noncompliance with vision care 3. Watkins K, Wood H, Schneider CR, Clifford R. Effectiveness of
guidelines in Latinos with type 2 diabetes mellitus: the Los Angeles implementation strategies for clinical guidelines to community
Latino Eye Study. Ophthalmology. 2006;113(8):1372-1377. pharmacy: a systematic review. Implement Sci. 2015;10:151.
5. Saadine JB, Fong DS, Yao J. Factors associated with follow-up eye exami- 4. Gold R, Nelson C, Cowburn S, Bunce A, Hollombe C, Davis J, Muench J,
nations among persons with diabetes. Retina. 2008,28(2):195-200. Hill C, Mital M, Puro J, Perrin N, Nichols G, Turner A, Mercer M, Jaworski V,
6. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, Howard C, Abiles E, Shah A, Dudl J, Chan W, DeVoe J. Feasibility and
McHugh SM, Weiner BJ. Enhancing the impact of implementation impact of implementing a private care system’s diabetes quality
strategies in healthcare: a research agenda. Front Public Health 2019,7(3). improvement intervention in the safety net: a cluster-randomized trial.
Implement Sci. 2015;10(1):83.
A32
Results from a randomized trial comparing strategies for helping A33
CHCs implement guideline-concordant cardioprotective care Main findings from the substance abuse treatment to HIV care
Rachel Gold1, Arwen Bunce2, Stuart Cowburn2, James V. Davis1, Joan (SAT2HIV) project: a type 2 effectiveness-implementation hybrid trial
Nelson2, Deborah J. Cohen3, James Dearing4, Michael A. Horberg5 Bryan Garner1, Stephen Tueller1, Steve Martino2, Heather Gotham3, Kathryn
1
Center for Health Research, Kaiser Permanente, Portland, OR, USA; Speck4, Michael Chaple5, Denna Vandersloot6, Michael Bradshaw1, Elizabeth
2
OCHIN, Portland, OR, USA; 3School of Medicine, Oregon Health & Ball1, Alyssa Toro1, Marianne Kluckmann1, Mathew Roosa7, James Ford8
Science University, Portland, OR, USA; 4Department of Communication, 1
RTI International, Research Triangle Park, NC, USA; 2Department of Psychiatry,
Michigan State University, East Lansing, MI, USA; 5Kaiser Permanente - Yale University, New Haven, CT, USA; 3Department of Psychiatry & Behavioral
Mid-Atlantic Permanente Research Institute, Rockland, MD, USA Sciences, Stanford University, Palo Alto, CA, USA; 4Public Policy Center,
Correspondence: Rachel Gold ([email protected]) University of Nebraska, Lincoln, Lincoln, NB, USA; 5NDRI, Inc, New York, NY,
Implementation Science 2020, 15(Suppl 2):A32 USA; 6The Alcohol and Drug Institute, University of Washington, Seattle, WA,
USA; 7Roosa Consulting, Syracuse, NY, USA; 8Center for Health Systems
Background Research and Analysis, University of Wisconsin, Madison, Madison, WI, USA
Statins can reduce cardiovascular disease (CVD) risk in patients with Correspondence: Bryan Garner ([email protected])
diabetes (DM), but prescribing often lags behind recommendations. We Implementation Science 2020, 15(Suppl 2):A33
compared how three increasingly intensive implementation support
strategies impacted community health centers’ (CHCs) adoption of elec- Background
tronic health record (EHR) clinical decision support tools targeting Improving the integration of substance use services within HIV service
guideline-concordant statin prescribing in DM [1-3]. The tools (the ‘CVD settings is an important public health concern [1]. To help understand
Bundle’) were adapted from a previously successful intervention [4]. how best to improve the integration of substance use services within
HIV service settings, the National Institute on Drug Abuse funded a type Background
2 effectiveness-implementation hybrid trial entitled the Substance Despite a growing body of evidence of implementation strategies for
Abuse Treatment to HIV Care (SAT2HIV) Project [2-3]. This presentation evidence-based care, a lack of sufficient detail often impedes suc-
focuses on the SAT2HIV Project’s main findings. cessful reproducibility and adequate evaluation of their efficiency [1].
Materials and Methods Development and specific description of operational and reprodu-
Using a cluster-randomized design, 39 HIV service organizations and cible strategies are crucial to efficiently and sustainably promoting
their staff were randomized to either implementation-as-usual (IAU) or uptake of evidence-based practice.
IAU plus Implementation & Sustainment Facilitation (IAU+ISF). As part Materials and Methods
of the IAU condition, staff received training, feedback, and coaching in In our effort to design a sustainable cervical cancer screening pro-
a motivational interviewing-based brief intervention (BI) for substance gram in Iquitos, Peru, we developed the Integrative Systems Practice
use. As part of the IAU+ISF, staff received the IAU strategy, as well as for Implementation Research (INSPIRE) Model, a multifaceted strategy
participated in external facilitation meetings with an ISF coach. Within that blends together existing theoretical frameworks and defines
each HIV service organization, eligible and consenting clients were ran- specific tools for use at each phase. INSPIRE is a participatory, itera-
domized to usual care (UC) or UC plus BI (UC+BI). The analytic sample tive process involving four phases: system understanding, finding le-
included 678 clients (82% follow-up rate), nested within 78 BI staff, verage, acting, and learning/adapting. Mixed methods are used to
nested within the 39 HIV service organizations. The preparation-phase create the shared understanding of the screening system and to fa-
outcome was staff time-to-proficiency (i.e., a staff-level measure of the cilitate identification of leverage points for change. A systems model-
number of days between completing the initial training and demon- ing tool was designed to compare alternative screening systems to
strating BI proficiency). Implementation-phase outcomes were: staff im- facilitate the decision-making process in a design workshop setting
plementation effectiveness (i.e., a staff-level measure of the consistency and working groups were formed to design new system processes.
and quality of BI implementation) and client substance use at follow-up Results
(i.e., a client-level measure of past-28 day primary substance use). Through phases 1-3 of the INSPIRE model, we engaged more than 90
Results multi-level stakeholders in the design of a new and improved screen
The ISF strategy reduced time-to-proficiency (β = - .66), but this reduction and treat system. Elaboration of system process maps through triangu-
was not significantly less (p < .05) than what was achieved by staff in the lation of the mixed-methods data served to create a shared reference
IAU condition. However, the ISF did significantly improved implementa- of the current system in participatory discussions. Significant leverage
tion effectiveness (β = .73, p < .001) beyond what was achieved in the opportunities were identified, including reducing fragmentation, ineffi-
IAU condition. Moreover, the ISF strategy did significantly improved the ciency, and a lack of standardization to increase women’s acceptability
BI’s effectiveness for reducing client substance use (β = -2.25, p < .05). of screening and adherence to continuum of care. A variety of interven-
Conclusions tions were evaluated and ultimately, stakeholders recommended adop-
Training, feedback, and coaching was sufficient for helping staff demonstrate tion of HPV testing/self-sampling to increase coverage and ablative
proficiency in a motivational interviewing-based BI for substance use. How- treatment of all HPV-positive women to reduce loss to follow up.
ever, the ISF strategy was found to help significantly improve implementa- Conclusions
tion effectiveness and help significantly reduce client substance use. Continued success in engagement of stakeholders in shared decision
Trial Registration ClinicalTrials.gov NCT03120598 making, including current development of a detailed implementation
plan using similar user-centered design, suggests that using a SPF in
References designing implementation strategies increases a sense of ownership
1. Sweeney S. Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and in the process, which may lead to more sustainable screening pro-
efficiency of integrating HIV/AIDS services with other health services: a systematic grams in LMIC compared with ‘top-down’ approaches.
review of evidence and experience. Sex Transm Infect. 2012;88(2):85-99.
2. Garner BR, Zehner M, Roosa MR., Martino S, Gotham HJ, Ball EL, Stilen P, Reference
Speck K, Vandersloot D, Rieckmann TR, Chaple M, Martin EG, Kaiser D, 1. Proctor EK, Powell BJ, McMillen JC. Implementation strategies:
Ford JH. Testing the implementation and sustainment facilitation (ISF) recommendations for specifying and reporting. Implement Sci.
strategy as an effective adjunct to the Addiction Technology Transfer 2013;8:139. doi:10.1186/1748-5908-8-139
Center (ATTC) strategy: study protocol for a cluster randomized trial.
Addict Sci Clin Pract. 2017;12(1):32. doi:10.1186/s13722-017-0096-7.
3. Garner BR, Gotham HJ, Tueller SJ, Ball EL, Kaiser D, Stilen P, Speck K, A35
Vandersloot D, Rieckmann TR, Chaple M, Martin EG, Martino, S. Testing the A secondary analysis of longitudinal state-level support for school-
effectiveness of a motivational interviewing-based brief intervention for based health centers mental health services
substance use as an adjunct to usual care in community-based AIDS service Tatiana Bustos, Amy Drahota, Kaston Anderson-Carpenter
organizations: Study protocol for a multisite randomized controlled trial. Ad- College of Social Science, Michigan State University, East Lansing, MI,
dict Sci Clin Pract. 2017;12(1): 31. doi:10.1186/s13722-017-0095-8. USA
Correspondence: Tatiana Bustos ([email protected])
A34
The Integrative Systems Practice for Implementation Research Background
(INSPIRE) model: application to context-appropriate design of a More than 20% of children in the U.S. experience mental health diffi-
cervical cancer screening program in the Peruvian Amazon culties, with only about 30% receiving adequate mental health (MH)
Valerie Paz-Soldan1, Magdalena Jurczuk1, Margaret Kosek2, Anne Rositch3, Graciela treatment. Moreover, MH service disparities are disproportionate
Meza4, Prajakta Asdul5, Laura Nervi6, J. Kathleen Tracy7, Javier Vasquez4, Renso among children who live in low-income areas [1-2]. School-based
Lopez4, Reyles Rios8, Joanna Brown9, Sandra Soto9, Patti Gravitt7 health centers (SBHCs), a comprehensive service delivery model inte-
1
Global Community Health and Behavioral Sciences, Tulane University, New grating physical and MH services within school settings, reduce
Orleans, LA, USA; 2Infectious Diseases and International Health, University of healthcare access barriers by functioning as medical centers for chil-
Virginia, Charlottesville, VA, USA; 3Department of Epidemiology, Johns dren in low-income areas. While many SBHCs in the U.S. offer some
Hopkins University, Baltimore, MD, USA; 4Facultad de Medicina Humana, type of MH service, not all centers are equipped to provide needed
Universidad Nacional de la Amazonia Peruana, Loreto, Peru; 5National Cancer MH services. MH service variations may be attributed to state-based
Institute, Bethesda, MD, USA; 6College of Population Health, University of policies, including funding, oversight support and standards. These
New Mexico, Albuquerque, NM, USA; 7University of Maryland School of outer contextual factors are thought to influence MH service
Medicine, Baltimore, MD, USA; 8Hospital Apoyo Iquitos “Cesar Garayar García”, provision by contributing to expansion and sustainment of services
Coronel, Portugal; 9Asociación Benéfica Prisma, Buenos Aires, Argentina over time [3-7]. This study aimed to examine how state-based outer
Correspondence: Valerie Paz-Soldan ([email protected]) contextual variables influence the number of SBHC-reported MH ser-
Implementation Science 2020, 15(Suppl 2):A34 vices over time.
Materials and Methods positive shifts in individuals’ contemplation and intention for imple-
The external policy and incentive domains of the Consolidated Framework mentation [4]. Historically, pre-implementation strategies have pro-
for Implementation Research (CFIR) were used to organize the secondary duced low implementation outcomes, especially when lacking a strong
data analysis of the State Policy Survey, a survey administered to policy- theoretical foundation [5]. The purpose of this study was to collect
makers knowledgeable of SBHC criteria within each state. Specifically, stakeholder feedback on the development of a blended pre-
state-based policymakers reported state support for SBHC funding, over- implementation strategy, Beliefs and Attitudes for Successful Imple-
sight and policies over four time-points: 2005, 2008, 2010, and 2014, while mentation in Schools for Teacher (BASIS-T), to precede an evidence-
SBHC personnel reported the number of MH services at these time-points. based training in classroom management.
A total of 4,232 SBHCs within 41 states were included in the study. To ac- Materials and Methods
count for inter-dependent groups of SBHCs within states, a linear mixed Twenty-two teachers and support staff from three Midwest school dis-
model analysis (LMM) was conducted to identify key variables within the tricts of diverse urbanicity engaged in a 3.5 hour demonstration of
domain of external policy and incentives that were significantly related to BASIS-T, which is grounded in the Theory of Planned Behavior [4], thus
the number of SBHC-reported MH services from 2005-2014. targeting teachers’ attitudes, social norm perceptions, and self-efficacy
Results beliefs. Throughout the demonstration, teachers rated each segment
Results indicated significant variations in the number of SBHC- for its acceptability and impact on shifting beliefs and attitudes. Partici-
reported MH services, accounted by state and time. Notably, most pants completed pre- and post-ratings of their own beliefs, attitudes,
outer contextual variables, with the exception of state general funds and intentions, engaged in a nominal group process to elicit feedback
on substance use treatment and referral services, were significantly for revising the strategies, and answered open-ended questions.
associated with more MH services over time. Results
Conclusions Participant feedback from nominal group processes highlighted
Findings suggest that there are significant relationships between ex- modifications to the pre-implementation strategy to improve its im-
ternal policies and the number of MH services being delivered by pact on beliefs, attitudes and implementation. Participants identified
SBHCs. However, these outer contextual variables had differential im- a need for more evocative and engaging activities to better encode
pacts depending on MH service type. the importance of EBP implementation, bolster their self-efficacy, and
address social norm perceptions. Average ratings indicated BASIS-T
References content was highly impactful and acceptable. Relatively lower-rated
1. Bains RM, Diallo AF. Mental health services in school-based health cen- segments were considered for revision.
ters: systematic review. J Sch Nurs. 2016; 32(1):8-19. doi: 10.1177/ Conclusions
1059840515590607 Pre-implementation strategies represent potentially useful techniques
2. Brown MB, Bolen LM. School-based health centers: strategies for meeting for aligning providers’ beliefs and attitudes prior to implementation
the physical and mental health needs of children and families. Psychol to facilitate better adoption. Stakeholder feedback is an effective
Sch. 2003;40(3):279-287. method for informing the development of these strategies. The re-
3. Doll B, Nastasi BK, Cornell L, Song SY. School-based mental health ser- sults of this demonstration study provided several recommendations
vices: Definitions and models of effective practice. J Appl Sch Psychol. and guidelines for how to build effective, school-based pre-
2017;33(3):179-194. doi: 10.1080/15377903.2017.1317143 implementation strategies to boost EBP adoption.
4. Sprigg SM, Wolgin F, Chubinski J, Keller K. School-based health centers: a
funder’s view of effective grant making. Health Aff. 2017;36(4):768. doi: References
10.1377/hlthaff.2016.1234 1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of
5. Anyon Y, Moore M, Horevitz E, Whitaker K, Stone S, Shields JP. Health evidence-based practice implementation in public service sectors. Adm
risks, race, and adolescents’ use of school-based health centers: policy Policy Ment Health. 2011;38(1):4-23.
and service recommendations. J Behav Health Serv Res. 2013;40(4):457– 2. Cook CR, Lyon AR, Kubergovic D, Wright DB, Zhang Y. A supportive
468. doi: 10.1007/s11414-013-9356-9. beliefs intervention to facilitate the implementation of evidence-based
6. Price OA. School-centered approaches to improve community health: practices within a multi-tiered system of supports. Sch Ment Health.
Lessons from school-based health centers. Economic Studies at Brook- 2015;7(1):49-60.
ings. 2016:5:1-17. 3. Stokke K, Olsen NR, Espehaug B, Nortvedt MW. Evidence based practice
7. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. beliefs and implementation among nurses: a cross-sectional study. BMC
Fostering implementation of health services research findings into Nurs. 2014;13(1):8.
practice: a consolidated framework for advancing implementation 4. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis
science. Implement Sci. 2009; 4:50. doi:10.1186/1748-5908-4-50. Process. 1991;50(2):179-211.
5. Yoon KS, Duncan T, Lee SW, Scarloss B, Shapley KL. Reviewing the
evidence on how teacher professional development affects student
A36 achievement. Issues & Answers. REL 2007-No. 033. Regional Educational
Teacher perspectives on the development of the Beliefs and Laboratory Southwest (NJ1). 2007.
Attitudes for Successful Implementation in Schools for Teachers
(BASIS-T)
Andrew Thayer1, James Merle1, Madeline Larson1, Jenna A37
McGinnis1, Clayton Cook1, Aaron Lyon2 Randomized trial to optimize a brief online training and
1
Department of Educational Psychology, University of Minnesota, Twin consultation strategy for measurement-based care in school
Cities, Minneapolis, MN, USA; 2Department of Psychiatry and Behavioral mental health
Sciences, University of Washington, Seattle, WA, USA Aaron Lyon, Freda F. Liu, Jessica I. Coifman, Heather Cook, Kevin
Correspondence: Andrew Thayer ([email protected]) King, Kristy Ludwig, Amy Law, Shannon Dorsey, Elizabeth McCauley
Implementation Science 2020, 15(Suppl 2):A36 University of Washington, Seattle, WA, USA
Correspondence: Aaron Lyon ([email protected])
Implementation barriers exist at all levels of service provision within an
organization, yet implementation is ultimately dependent on individ- Background
uals [1]. Converging lines of research have demonstrated that individual Pragmatic and streamlined implementation strategies are necessary
beliefs and attitudes are associated with implementation outcomes [2- for efficient quality improvement [1]. Training and post-training con-
3]. Active-implementation strategies targeting individuals can be effect- sultation are cornerstone implementation strategies [2], but are often
ive for preventing implementer drift, yet they occur after implementa- lengthy and resource-intensive [3]. Further, few studies have evalu-
tion is underway. Pre-implementation strategies occurring prior to ated their mechanisms of action [4]. Development and optimization
uptake may be effective in aligning beliefs and attitudes resulting in of these strategies requires attention to (a) user experience, to ensure
that the strategy is compelling and easy to use; and (b) strategy ef- children’s access to specialty mental health care), and identifies col-
fectiveness, to ensure that the strategy influences its targeted mech- laboration strategies and challenges at each stage.
anisms of action. This presentation will present findings from a Results
project that designed and tested a brief, multifaceted online training Fidelity to initial mental health screening and assessment was high
and post-training consultation strategy to target each strategy’s puta- and attributed to service co-location, collaborative workflow plan-
tive mechanisms of action and support measurement-based care ning, and linked data systems. However, fidelity to referral/treatment
(MBC) practices among school-based mental health clinicians. components was low and associated with case planning challenges,
Materials and Methods contract disruptions, and workforce shortages.
Iterative development of the online training and post-training con- Conclusion
sultation strategies involved gathering stakeholder input via four Our findings suggest that fidelity breakdowns at any point in the ser-
rounds of usability testing and two group cognitive walkthrough ses- vice cascade can negatively affect clients’ access to services, espe-
sions with representative clinician users. This culminated in random- cially during key service transitions. Implementing these models
ized trial in which 77 geographically diverse school-based mental likely depends on cross-system collaboration approaches that align
health clinicians were randomized to (1) implementation as usual front-line practice and agency operations at each stage of the model.
(IAU; no training or consultation, n = 40), or to online training plus
one of three consultation dosages: (2) 2 weeks, (3) 4 weeks, or (4) 8
weeks. Consultation included live consultation video calls (once every
two weeks) and asynchronous message board discussion. Following A39
training, training mechanisms (knowledge, attitudes, skill), consult- A pragmatic method for costing implementation strategies using
ation mechanisms (collaboration, responsiveness, accountability), and the time-driven activity-based costing
self-reported clinician MBC practices were tracked for 16 weeks. Zuleyha Cidav1, Jeffrey Pyne2, Geoffrey Curran2, David Mandell1, Rinad S.
Results Beidas1, Jennifer Mautone1, Ricardo Eiraldi1, Steven Marcus1
1
Preliminary analysis (multilevel modeling) showed that online train- University of Pennsylvania, Philadelphia, PA, USA; 2University of
ing led to an immediate increase in MBC knowledge relative to con- Arkansas for Medical Sciences, Little Rock, AR, USA
trols (β= .06, p<.05). Following consultation, participants Correspondence: Zuleyha Cidav ([email protected])
demonstrated greater growth in self-reported MBC skills (β=.028, p< Implementation Science 2020, 15(Suppl 2):A39
0.01) and attitudes (e.g., perceived benefit of MBC, β=.028, p<0.05)
and superior MBC practices (e.g., use of standardized and individual- Background
ized assessments, β=.013, .032, respectively, p<0.01). Clear consult- Strategies to implement evidence-based practices consume
ation dosage effects have yet to emerge from preliminary analyses scarce resources and incur costs. Although critical for decision
with 16 (of 32) weeks of follow-up data. It is possible that the con- makers with constrained budgets and limited resources, such
sultation groups may become more disparate with longer follow-up. resource use and cost information are not typically reported.
Conclusions This is at least partly due to a lack of clearly defined and stan-
Few studies have examined implementation strategy mechanisms of dardized costing methods for use in implementation science.
action. The current findings suggest that the online training and con- This study presents a pragmatic approach to systemically esti-
sultation package influenced many of its target mechanisms and also mating resource use and costs of implementation strategies
lead to higher MBC practices among school mental health clinicians using a well-established business accounting system. The
than IAU. Thus far, preliminary results suggest that shorter durations method is demonstrated by estimating the first-year implemen-
of consultation may be comparable to longer durations. tation costs of a group-based cognitive behavioral therapy
program for students with externalizing disorders in six Phila-
References delphia schools.
1. Lewis CC, Klasnja P, Powell BJ, Tuzzio L, Jones S, Walsh-Bailey C, Weiner B. Materials and Methods
From classification to causality: advancing understanding of mechanisms Time-driven activity-based costing (TDABC) is combined with the
of change in implementation science. Front Public Health. 2018;6:136. existing guidelines for implementation strategy specification and
2. Lyon AR, Pullmann MD, Walker SC, D’Angelo G. Community-sourced reporting. Implementation protocol, measures, project notes and
intervention programs: Review of submissions in response to a statewide key personnel interviews were used to map the implementation
call for “promising practices.” Adm Policy Ment Health. 2017;44(1):16-28. process by specifying the strategies with their actors, specific ac-
3. Olmstead T, Carroll KM, Canning-Ball M, Martino S. Cost and cost- tion steps, temporality, and dose. The dose is defined for each
effectiveness of three strategies for training clinicians in motivational action step as the person-hours invested in its completion, and
interviewing. Drug Alcohol Depend. 2011;16(1-3):195-202. accounts both for frequency and intensity of the action step. Im-
4. Williams NJ. Multilevel mechanisms of implementation strategies in plementation strategy dose is the sum of person-hours on each
mental health: Integrating theory, research, and practice. Adm Policy action step that constitute the strategy. Project resources are
Ment Health. 2016;43(5):783-798. identified, and the price per unit person-hour is calculated as per
the TDABC. Costs of action steps, strategies and implementation
project is reported from a payer perspective.
A38 Results
Implementing service cascade models with fidelity: a case study of Estimated total cost was $63,842; $10,640 per school. The largest cost
cross-system collaboration strengths and challenges incurred was for the communication efforts ($30,691), which involved
Alicia Bunger, Christy Kranich, Susan Yoon, Lisa Juckett in-person meetings, phone calls, and email exchanges. Next largest
Ohio State University, Columbus, OH, USA costs were for the stakeholder engagement, consultation/coaching,
Correspondence: Alicia Bunger ([email protected]) and supervision, which comprised 19%, 15%, 12% of total costs, re-
Implementation Science 2020, 15(Suppl 2):A38 spectively. Assessment/evaluation and training constituted the smal-
lest costs, at 4% and 3% respectively.
Background Conclusions
Service cascade models move individuals across systems through a This method allows for inclusion of implementation costs in the ef-
sequence of screening, assessment, referral, treatment, and monitor- forts of strategy specification, tracking and reporting. It serves as a
ing activities. Successful implementation depends on strong collabor- pragmatic tool to operationalize the conduct of the implementation
ation and change across systems, but these models have received activities, track the resources consumed and estimate associated
limited empirical attention. costs. It could facilitate the routine incorporation of cost analysis and
Materials and Methods economic evaluations into implementation research. It provides
This case study examines fidelity of a cascade model implemented granular cost information which could be used to identify and ad-
across child welfare and mental health systems (intended to improve dress the inefficiencies in the implementation process.
A40 These relationships, which serve as the vehicle for translating science
Shared goal, different languages: communication between into practice, are often understated and under examined. This panel
implementation researchers and social entrepreneurs presents two initiatives designed to implement best practices within a
Enola Proctor1, Rachel Tabak1, Cole Hooley1, Virginia McKay1, Emre Toker2 large child welfare service system focused on all three perspectives: sys-
1
Brown School, Washington University in St. Louis, St. Louis, MO, USA; tem leadership, presented by Kimberly Giardina, MSW; implementation
2
Arizona State University, Tempe, AZ, USA science, presented by Greg Aarons, PhD., and intermediary implemen-
Correspondence: Enola Proctor ([email protected]) tation, presented by Brent Crandal, PhD., and Melissa Bernstein, PhD.
Implementation Science 2020, 15(Suppl 2):A40 Materials and Methods
The two system changes that will provide context for this panel include,
Background first, the Advancing California’s Trauma Informed Systems (ACTS) Initia-
Implementation science and social entrepreneurship share a common tive. ACTS was developed to create collaborative partnerships with child-
objective of maximizing the uptake of new practices and innovations. welfare county leaders across California to advance trauma- and
Both disciplines offer complementary tools which could be leveraged evidence-informed system change through implementation planning,
to improve the ultimate impact of innovations, their scale-up, and sus- followed by technical assistance, training, outcome monitoring, and sus-
tainment. For example, entrepreneurship’s focus on market assessment, tainment planning. Second, the Community-Academic Partnerships for
financial outlook, etc. can be coupled with implementation science use the Translational Use of Research Evidence (CAPTURE) project explores
of data, models, and methods. However, communication between how research can be used to improve child welfare policy, programs and
these fields has been limited. To explore how these complementary practices through a partnership between the University of California at
groups might learn from each other, this paper presents data about San Diego (UCSD) and the County of San Diego, Health and Human Ser-
how implementation researchers understand and respond to the kinds vices Agency, Child Welfare Services.
of questions entrepreneurs ask about innovation roll-out. Results
Materials and Methods Using a mixed-methods research design, CAPTURE examines the pro-
We conducted one-on-one cognitive interviews [1] with 15 dissemination cesses that shape instrumental and conceptual use of research evi-
and implementation researchers recruited from training programs in imple- dence (URE) and investigates how change mechanisms influence URE.
mentation science to capture their ability to understand and answer key Conclusions
questions from entrepreneurs and refine a tool to support dialogue. Partici- Guided by the Exploration, Preparation, Implementation, Sustainment
pants were guided through the tool item-by-item. Prompts from the inter- (EPIS) implementation framework, we will highlight the translation of
viewer helped identify problems with question clarity and comprehension. implementation science into concrete strategies used to create
A summary of the responses was developed to identify problematic ele- change within a child welfare system, including leadership engage-
ments and identify revisions necessary to improve clarity. The discussions ment, team development, data utilization, stakeholder involvement,
tapped the researchers’ perception of demand for the innovation, estimates quality improvement methods, and the use of consultation.
of benefit, knowledge of who would pay, sustainment challenges, and com-
fort working with business and entrepreneurial partners. Reference
Results 1. National Implementation Research Network Active Implementation
Implementation researchers understood and were able to answer Practice and Science (Issue Brief No. 1). 2016. https://nirn.fpg.unc.edu/
questions about the problem their innovation seeks to solve, their sites/nirn.fpg.unc.edu/files/resources/NIRN-Briefs-1-
roll-out plans, and stakeholders and team members involved. They ActiveImplementationPracticeAndScience-10-05-2016.pdf
reported the following types of questions as easy to understand but
hard to answer: who would pay for the innovation, numbers of
people who would benefit, and how to sustain the innovation once A42
adopted. Researchers varied in their comfort with technology sup- A multiple case study of a tailored approach to implementing
ports and business/entrepreneurial partnerships. measurement-based care for depression
Conclusions Byron J. Powell1, Meredith Boyd2, Hannah Kassab3, Cara C. Lewis4
1
Entrepreneurial partnerships can provide important supports to successful Brown School, Washington University in St. Louis, St. Louis, MO, USA;
2
implementation and sustained delivery of interventions, including technol- University of California, Los Angeles, Los Angeles, CA, USA; 3Ohio
ogy supports, market analysis, and financial investment. Implementation University, Athens, OH, USA; 4Kaiser Permanente Washington Health
researchers understand the types of questions entrepreneurs pose about Research Institute, Seattle, WA, USA
their projects, but find those questions difficult to answer, suggesting the Correspondence: Byron J. Powell ([email protected])
importance of establishing interface between these fields. Tools to pre- Implementation Science 2020, 15(Suppl 2):A42
pare researchers to interact with entrepreneurs could take these factors
into account to facilitate communication between audiences. Background
Tailoring implementation strategies to site-specific determinants (bar-
Reference riers and facilitators) is a promising way of improving implementation
1. Willis GB. Cognitive interviewing: A tool for improving questionnaire and clinical outcomes [1]. However, more empirical work is needed to
design. Thousand Oaks, CA: Sage Publications, Inc; 2005. determine whether tailored approaches to implementation are more
effective than standard multifaceted strategies, and to develop optimal
methods for linking implementation strategies to identified determi-
A41 nants [2]. The NIMH-funded “Implementing Measurement-Based Care
Making it happen: implementation efforts for systems level change (iMBC) for Depression in Community Mental Health” study addressed
in child welfare these gaps by comparing a standard multifaceted strategy to a tailored
Melissa Bernstein1, Brent Crandal1, Gregory Aarons2, Kimberly Giardina3 approach to implementation in a dynamic cluster randomized trial [3].
1
Rady Children’s Hospital, San Diego, CA, USA; 2University of California This study draws upon data from the intervention group (i.e., tailored
San Diego, San Diego, CA, USA; 3County of San Diego Health and condition) of the iMBC study to 1) describe how clinics tailored imple-
Human Services, San Diego, CA, USA mentation strategies to site-specific barriers, and 2) evaluate the extent
Correspondence: Melissa Bernstein ([email protected]) to which the implementation strategies they used could plausibly
Implementation Science 2020, 15(Suppl 2):A41 address identified determinants.
Background The six clinics in the tailored condition were compared to each other
If diffusion is “letting it happen,” and dissemination is “helping it using a multiple case study design. Descriptions of each clinic’s approach
happen,” implementation is the business of “making it happen” [1]. To to tailoring, the determinants they attempted to address, and the imple-
make change happen within large systems, relationships between im- mentation strategies they used were derived from recordings and notes
plementation scientists, intermediaries, and system leaders are created. from implementation team meetings across five months. Determinants
were deductively coded using the Consolidated Framework for Imple- adaptation process [1]. We also conducted a systematic review of
mentation Research (CFIR) [4] and implementation strategies were de- adapted interventions and described reasons for adaptation accord-
ductively coded using an established taxonomy [5]. Plausibility of ing to previously identified categories [2-3]. These studies have
linkages between barriers and strategies was determined in two ways. shown that while many examples and adaptation models exist, they
First, two authors independently rated plausibility using a 5-point Likert provide only limited guidance on how to make decisions about what
scale. Second, each strategy-determinant linkage was compared to the should change and what should remain the same.
results of a previous study that established preliminary linkages between Results
CFIR determinants and implementation strategies [6]. We present a framework based on the Intervention Mapping protocol
Results that provides step-by-step guidance on selection and adaptation of
Four of the six clinics prioritized barriers identified quantitatively during the EBIs [4-5]. The process includes the development of a logic model of
needs assessment phase of the study, and explicitly selected implementation change (LMC) based on the community assessment. An LMC is a dia-
strategies to address them. Clinics reported using an average of 39 imple- gram of what that describes the relationship between changes needed
mentation strategies, which were categorized into 26 of the 68 discrete imple- in determinants, behavior and environment to bring about improve-
mentation strategies identified by Powell et al. [7]. Plausibility of the linkages ments in health and quality of life. The LMC is then compared with the
between barriers identified and strategies selected will also be presented. basic features of available EBIs (determinants addressed, resources
Conclusions needed, etc.) to assess potential fit with the new population or setting.
This study contributes to implementation science and practice by Following selection, planners further examine the internal logic of the
highlighting strengths and weaknesses of community mental health EBI including the behaviors and environmental conditions that were
clinics’ approaches to tailoring implementation strategies, and by the targets of the original EBI, the determinants addressed, and the
suggesting ways in which methods for tailoring could be improved. change methods and/ or strategies used. Planners then compare the
Trial registration: ClinicalTrials.gov NCT02266134 EBI features to the LMC to determine what needs to be adapted while
maintaining change methods used in the original intervention since
References these often represent the intervention’s core elements.
1. Baker R, Comosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Conclusions
Robertson N, Wensing M, Fiander M, Eccles MP, Godycki-Cwirko M, van We describe the development and testing of an online tool (I M
Lieshout J, Jäger C. Tailored interventions to address determinants of ADAPT) for finding and adapting evidence based interventions for
practice. Cochrane Database Syst Rev. 2015;4(CD005470):1-118. cancer control. We also describe how we are applying Intervention
doi:10.1002/14651858.CD005470.pub3 Mapping and the online tool in a project to improve the use of inter-
2. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, ventions from the National Cancer Institute’s Research Tested Inter-
Mandell DS. Methods to improve the selection and tailoring of vention Programs (RTIPs) resource.
implementation strategies. J Behav Health Serv Res. 2017;44(2):177-194.
doi: 10.1007/s11414-015-9475-6. References
3. Lewis CC, Scott K, Marti CN, Marriott BR, Kroenke K, Putz JW, Mendel P, 1. Escoffery C, Lebow-Skelly E, Udelson H, Boing E, Fernandez M, Mullen PD.
Rutkowski S. Implementing measurement-based care (iMBC) for depression A scoping study of program adaptation frameworks for evidence-based
in community mental health: a dynamic cluster randomized trial study interventions. Transl Behav Med. 2019;9(1):1-10. doi: 10.1093/tbm/ibx067.
protocol. Implement Sci. 2015;10:127. doi:10.1186/s13012-015-0313-2. 2. Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R.,
4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Hartman M, Fernandez ME, Mulled PD. A systematic review of adapted
Fostering implementation of health services research findings into evidence-based interventions globally. Implement Sci. 2018;13:125.
practice: a consolidated framework for advancing implementation 3. Stirman SW, Miller, CJ, Toder K, Calloway A. Development of a framework
science. Implement Sci. 2009;4:50. and coding system for modifications and adaptions of evidence-based
5. Boyd MR, Powell BJ, Endicott D, Lewis CC. A method for tracking interventions. Implement Sci. 2013;8:65.
implementation strategies: An exemplar implementing measurement- 4. Bartholomew Eldredge LK, Markham CM, Ruiter RAC, Fernández M.E, Kok
based care in community behavioral health clinics. Behav Ther. G, Parcel GS. Planning health promotion programs: an Intervention
2018;49:525-537. doi:10.1016/j.beth.2017.11.012. Mapping approach, 4th ed. San Francisco, CA: Jossey-Bass; 2016.
6. Damschroder LJ, Waltz TJ, Abadie B, Powell BJ. Choosing implementation 5. Highfield L, Hartman MA, Mullen PD, Rodriguez SA, Fernandez ME,
strategies to address local contextual barriers. Implement Sci. 2018;13(Suppl3: Bartholomew LK. Intervention Mapping to adapt evidence-based interven-
A76):37-38. tions for use in practice: increasing mammography among African Ameri-
7. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, can women. Biomed Res Int. 2015;160103. doi:10.1155/2015/160103.
Glass JE, York JL. A compilation of strategies for implementing clinical
innovations in health and mental health. Med Care Res Rev.
2012;69(2):123-157. doi:10.1177/1077558711430690. A44
Co-creation of change in policy and practice: the Community
Academic Partnership for Translational Use of Research Evidence
A43 (CAPTURE)
Systematic adaptation of evidence-based interventions: an Gregory Aarons1, Kimberly Giardina2, Danielle Fettes3, Margo Fudge2, the
intervention mapping approach CAPTURE Steering Committee
Maria E. Fernández1, Cam Escoffery2, Maya Foster1, Patricia Mullen1 1
University of California, San Diego, San Diego, CA, USA; 2Child Welfare
1
University of Texas School of Public Health, Houston, TX, USA; 2Rollins Services - County of San Diego Health & Human Services Agency, San
School of Public Health, Emory University, Atlanta, GA, USA Diego, CA, USA; 3Child and Adolescent Services Research Center, San
Correspondence: Maria E. Fernández ([email protected]) Diego, CA, USA
Implementation Science 2020, 15(Suppl 2):A43 Correspondence: Gregory Aarons ([email protected])
Background
Evidence-based public health translation of research to practice is essen- Background
tial to improving the public’s health. Adaptation of evidence-based inter- Youths in the child welfare (CW) system face a myriad of poor out-
ventions (EBIs) to promote health and prevent disease is an essential comes with regard to social/emotional development and behavioral
process for implementation and dissemination research and practice. health. While there is research evidence relevant for CW services, in-
Challenges faced in practice include identifying EBIs that are suitable for tegration and use of such evidence in policy, planning, and service
new populations and settings and adapting them to fit needs. delivery is limited. The Community Academic Partnership for Transla-
Materials and Methods tional Use of Research Evidence (CAPTURE), is a community-
A recently published scoping review by our team summarized 13 academic partnership to increase use of research evidence in policy,
adaptation frameworks and identified common steps to guide the programs, and practice. Guided by the Exploration, Preparation,
Implementation, Sustainment (EPIS) framework, CAPTURE engages national leaders (mental health operations, suicide prevention, and
outer context system level stakeholders, inner context organizational technology). Results of themes will be presented, with a focus on
stakeholders and academic partners in a bidirectional partnership. how to conduct strategic implementation research planning in col-
The aims of CAPTURE are: 1) To establish and test the use of a part- laboration with a variety of stakeholders.
nership model to increase use of research evidence (URE) in policy, Conclusions
program, and practice, and 2) To identify key mechanisms by which Strategic implementation research planning is critical to developing
CAPTURE operates including - but not limited to - cultural exchange a research plan that both helps the healthcare system move forward
between researchers and community partners, leadership and in its goals and advances implementation science.
organizational change, and use of quality improvement methods to
test and put goals into practice. Reference
Materials and Methods 1. Stetler CB, Mittman BS, Francis J. Overview of the VA Quality
Mixed-methods will provide a detailed and nuanced understanding of Enhancement Research Initiative (QUERI) and QUERI theme articles:
the process of collaboration and URE in a large public sector service sys- QUERI Series. Implement Sci. 2008;3:8.
tem, and describe the complexity of instantiating URE across system and
organization levels. Quantitative data assessing cultural exchange, leader-
ship and climate for URE are being collected by online surveys of pro- A46
viders and stakeholders. Qualitative methods include interviews, focus Bridging the implementation research to practice gap: exploring
groups, observation of CAPTURE meetings, and document review. collaboration and solutions between researchers, policy-makers
Results and funders, implementation supports and implementing
Community-academic partnerships are a promising approach to im- organisations
proving collaboration and integrating research evidence in decision Chair: Jacquie Brown1, Discussant: Aaron Lyon2, Panelists: Byron Powell3,
making for policy and practice. Jenna McWilliam4, Arthur Evans5
1
Conclusions Families Foundation, Hilversum, The Netherlands; 2University of
A better understanding of ways to develop and establish Washington, Seattle, WA, USA; 3Brown School, Washington University in
community-academic partnerships can help to promote their use in St. Louis, St. Louis, MO, USA; 4Triple P International, Brisbane,
other settings. Queensland, Australia; 5American Psychological Association, Washington,
DC, USA
Correspondence: Jacquie Brown (jacquie.brown@familiesfoundation.
A45 net)
Developing a strategic implementation research plan within an Implementation Science 2020, 15(Suppl 2):A46
integrated healthcare system
Sara J. Landes1,2, JoAnn E. Kirchner1,2, Mark Bauer1,3, Christopher Background
Miller1,3, Mona Ritchie1,2, Jeffrey L. Smith1,2 The intersection between the various perspectives is critical in influ-
1
Behavioral Health QUERI, Little Rock, AR, USA; 2Central Arkansas encing the extent to which implementation science can be translated
Veterans Healthcare System, Little Rock, AR, USA; 3VA Boston Healthcare into implementation practice. At this juncture of the development of
System, Boston, MA, USA implementation science it is critical that we develop the capacity to
Correspondence: Sara J. Landes ([email protected]) integrate the knowledge from each perspective to ensure that imple-
Implementation Science 2020, 15(Suppl 2):A45 mentation science achieves its ultimate goal of “bridging the gap”.
Collaboration will promote the awareness, understanding and ability
Background to consider all perspectives whilst further developing the science and
The US Department of Veterans Affairs (VA) Quality Enhancement Re- supports for application of implementation science.
search Initiative (QUERI) funds quality improvement and program Having explored the “constructive tension” between implementation
evaluation studies to support implementation and evaluation efforts researchers/intermediaries/purveyors/implementing organisations at
needed to improve healthcare for our nation’s veterans [1]. QUERI GIC 2017, at which themes were identified that reflected the chal-
funds programs that focus on areas of care and/or implementation lenges between implementation science and implementation prac-
strategies. The Behavioral Health QUERI program, one of 15 currently tice, the themes were furthered discussed at Global Evidence and
funded programs, has developed a structured method for strategic Implementation Symposium, 2018. Greater insight was generated
planning to match program priorities and implementation research into the challenges for each perspective and how they might be
projects with priorities of stakeholders and the healthcare system. addressed.
Embedded with the organizational structure of the Behavioral Health Materials and Methods
QUERI are a Stakeholder Council (SC) comprised of veterans of all The panel proposed for SIRC 2019, with active participation from the
eras, family members, providers, and local and regional leadership; attendees, will build on the previous discussions. This participatory
and a Strategic Advisory Group (SAG) comprised of national health- discussion will focus on solutions, exploring and developing potential
care leaders inside and outside of VA and veteran representatives. next steps to address identified barriers, and build on existing part-
Materials and Methods nerships across perspectives.
The strategic planning methods are iterative and include stakeholders The session will open with a brief reference to information gathered
from multiple levels of the national health care system (e.g., providers, at the previous two sessions. Each panelist will then offer thoughts
leadership at various levels, veterans, and subject matter experts). The from their perspectives on what might promote greater integration
strategic planning methods include identification of priorities of the between each perspective.
healthcare system, creation of a planning committee, development of a Results
key stakeholder interview, and identifying key informants. Stakeholder Following the brief presentations, the floor will be opened for a facili-
interviews are conducted across multiple layers of the organization tated discussion framed around three questions:
(e.g., veterans, local medical center, network level, and national leader- 1) What are the actions we can take to increase the collaboration be-
ship). Qualitative data are synthesized across key question domains tween the various contributors to implementation science and
and findings are matched to existing initiatives and/or research oppor- practice?
tunities by the planning committee. Priorities and potential projects are 2) What are the barriers you experience that might interfere with
identified and prioritized with the SC and SAG. These are vetted with a these actions?
sample of the initial key informants. 3) What are the opportunities/enablers you could use to promote
Results collaborative development of implementation science and practice?
Behavioral Health QUERI is currently conducting strategic planning. The session will close with a summary from the discussant.
VA priorities have been identified by the SC and SAG. Stakeholder in- Conclusions
terviews have been completed with three network leaders and three The objectives for the panel discussion include:
1) Increasing awareness of the needs, challenges and opportunities A48

for collaboration across perspectives The Department of Defense Practice-Based Implementation
2) Identifying priorities for research and partnerships for action Network: developing a framework for matching implementation
3) Confirm 3 areas for action for 2019 – 2021 strategies to barriers in complex healthcare systems
This is a single theme symposium with brief presentations from the Kimberly Pratt, Briana Todd, Angela Gray, Jorielle Houston
panelists followed by open-floor discussion. Psychological Health Center of Excellence, Defense Health Agency, Silver
Spring, MD, USA
A47 Correspondence: Kimberly Pratt ([email protected])
HealthLinks: evaluation challenges and learnings from three Implementation Science 2020, 15(Suppl 2):A48
organisational perspectives
Norm Good, Philippa Niven Background
CSIRO, Canberra, Australia Integrating evidence-based practices into clinical care is essential for mis-
Correspondence: Norm Good ([email protected]) sion readiness. However, research demonstrates, that despite the avail-
Implementation Science 2020, 15(Suppl 2):A47 ability of effective clinical interventions, service members frequently do
not receive guideline-concordant care that reflects the uptake of the
Background most recent scientific advancements [1]. To improve the access, quality,
Health care systems across the developed world are currently facing effectiveness, and efficiency of psychological healthcare for veterans and
a similar set of challenges. Populations are ageing and chronic ill- service members, the Department of Defense (DoD) established the
nesses are becoming more prevalent. A relatively small subset of Practice-Based Implementation (PBI) Network. The PBI Network engages
complex patients with chronic medical conditions account for a large healthcare administrators and providers in implementation pilots to
proportion of hospital re-admissions and consume a significant num- evaluate the feasibility and acceptability of DoD-wide implementation of
ber of hospital resources [1]. HealthLinks Chronic Care (HLCC) is an an evidence-based clinical practice, and develops solutions to any imple-
initiative undertaken by the Victorian Department of Health and Hu- mentation challenges that could impede successful uptake and sustained
man Services (DHHS) to provide a flexible funding model for partici- use of the practice change. The purpose of this presentation is to review
pating hospitals to convert projected inpatient costs towards new or the PBI Network’s process for mapping implementation strategies to bar-
improved patient centred models of care with the aim of reducing riers in the Military Health System (MHS).
unplanned hospital admissions. Materials and Methods
Materials and Methods This presentation will review a case study showcasing the PBI Net-
Six health services within Victoria, Australia have implemented a new work’s efforts to partner with DoD leadership, clinicians, and re-
or improved model of care unique to their health service needs and searchers to enhance access to evidence-based practices in the MHS.
the demographic profile of their patient population with an extra The case study will demonstrate the PBI Network’s use of Interven-
three acting as controls. DHHS contracted the Commonwealth Scien- tion Mapping (IM) [2] guided by the Consolidated Framework for Im-
tific and Industrial Research Organisation (CSIRO) to work on a plementation Research (CFIR) [3] and the Integrated Promoting
cosponsored system level evaluation of HLCC. The evaluation is Action on Research in Health Services (i-PARIHS) [4] frameworks to
based on the RE-AIM framework [2] and uses a comprehensive mixed develop an implementation program that strategically addresses the
methods approach including analysis of routinely collected hospital unique challenges facing military health providers and leaders.
data using a BACI design [3], a quality of life patient survey, work- Results
force interviews and costings data. Time constraints, multiple competing demands, and limited re-
Results sources [5-6] are significant barriers to the uptake and adoption of
The overall aim of the HLCC evaluation is to determine if flexible evidence-based practices in the MHS. Many implementation strat-
funding enables health services to develop and implement alterna- egies can be leveraged to overcome these barriers and effectively
tive models to inpatient acute care that provide better experiences implement evidence-based practices in the MHS.
and outcomes for patients with chronic conditions, at equal or lower Conclusions
cost. Implementation of evidence-based intervention is a complicated
Conclusions process. It has been suggested that implementation strategies should
This session will provide an overview of HLCC and aims to describe be selected and tailored to address the contextual needs of specific
key challenges and learnings of the trial from three different efforts; however, there is limited guidance as to how to do this. The
perspectives: PBI Network’s experiences provide insight into methodologies for
mapping implementation strategies to address the complex barriers
to implementation in the MHS which may have broad application to
DHHS - policy developers/funders: Implications of a pragmatic other similar complex healthcare systems.
approach to implementation will be discussed, including the
pros and cons of having implemented the trial at several
health services and allowing tailored intervention models. References
Health Services - implementers: Results from qualitative focus 1. Institute of Medicine. Treatment for Posttraumatic Stress Disorder in military
groups assessing workforce perceptions of key barriers and and veteran populations: final assessment. National Academies Press. 2014.
enablers to implementation will be presented. 2. Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Intervention mapping:
CSIRO - evaluators: The feasibility of evaluation frameworks in a designing theory and evidence-based health promotion programs
data driven environment will be discussed, including how data Mountain View: Mayfield Publishing Company; 2001.
lags, data availability and outcome measures impact reporting 3. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC.
and short-term policy decisions. Fostering implementation of health services research findings into
References science. Implement Sci. 2009;4:50.
1. Calver J, Brameld KJ, Preen DB, Alexia SJ, Boldy DP, McCaul KA. High-cost 4. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated
users of hospital beds in Western Australia: a population-based record framework for the successful implementation of knowledge into practice.
linkage study. Med J Aust. 2006;184:393–397. Implement Sci. 2016;11:33.
2. Glasgow, RE, Vogt, TM, Boles, SM. Evaluating the public health impact of 5. Cook JM, Dinnen S, Thompson R, Ruzek J, Coyne JC, Schnurr PP. A
health promotion interventions: the RE-AIM framework. Am J Public quantitative test of an implementation framework in 38 VA residential
Health. 1999;89:1322-1327 PTSD programs. Adm Policy Mental Health. 2016;42(4):462-473.
3. Underwood AJ. Beyond BACI: Experimental designs for detecting human 6. Sadeghi‐Bazargani H, Tabrizi JS, Azami‐Aghdash S. Barriers to evidence‐
environmental impacts on temporal variations in natural populations. based medicine: a systematic review. Journal of Eval Clin Pract.
Aust. J. Mar. Freshwater Res.1991;42:569-587 2016;20(6):793–802.
A49 A50
Optimizing public health interventions by using mechanistic How policy mandates for evidence-based practices filter into
evaluations: a case example from a school-based physical activity clinical routines: a mixed methods study
implementation trial Lorella Palazzo1, Peter Mendel2, Kelli Scott3, Cara C. Lewis1
Hopin Lee1, Nicole Nathan2, Kirsty Hope2, Luke Wolfenden2 1
Kaiser Permanente Washington Health Research Institute, Seattle, WA,
1
University of Oxford, Oxford, England, UK; 2University of Newcastle, USA; 2RAND Corporation, Santa Monica, CA, USA; 3Brown University
Newcastle, New South Wales, Australia School of Public Health, Providence, RI, USA
Correspondence: Hopin Lee ([email protected]) Correspondence: Lorella Palazzo ([email protected])
Public health implementation strategies often comprise of multiple com- Federal and state policies mandating evidence-based practice (EBP)
ponents that are combined and delivered as a complex intervention [1]. implementation in community mental health settings are increas-
Within a complex social-ecological system, there are multiple mechanisms ingly common. Measurement based care (MBC), the use of progress/
by which an intervention could have its effect on the distal implementa- outcome monitoring measures to guide treatment [1], is one such
tion outcome. To successfully implement health policies, the implementa- EBP mandated for use. It is crucial to understand the process by
tion strategy must collectively have a causal effect on the mechanisms which mandates are received by organizations, as contextual factors
that drive successful implementation [2]. Understanding these mecha- may influence mandate implementation success. The goal of this
nisms are critical to optimizing and scaling complex implementation strat- study is to employ mixed methods to characterize how a mandate
egies [3]. The study aim was to understand the mechanisms of a complex plays out in community mental health settings that implement MBC
implementation strategy on increasing physical activity minutes scheduled using the Patient Health Questionnaire-9 (PHQ-9).
by school teachers in New South Wales, Australia [4]. Materials and Methods
Materials and Methods We utilized data from a cluster randomized trial comparing tailored
We conducted a causal mediation analysis [5-6] of a cluster randomised vs standardized approaches to implementing the PHQ-9 in commu-
controlled trial conducted in 62 primary schools. Schools were randomly nity mental health settings in two states [2]. Qualitative data were
allocated to receive either a complex implementation strategy that in- collected through semi-structured interviews with community mental
cluded; obtaining executive support, provision of tools and resources, im- health clinicians (N = 36). Interview questions covered MBC domains
plementation prompts, reminders and feedback; or usual practice. The (e.g. clinicians’ attitudes towards and usage of MBC practices with
primary trial outcome was the average minutes of physical activity sched- the PHQ-9). Quantitative data were obtained from clinician surveys
uled by teachers across the school week at 12 months. We estimated and included demographics and contextual constructs (e.g. norms,
path specific effects and average indirect and direct effects of the imple- structures, processes) known to influence implementation.
mentation strategies through four putative mechanisms. Results
Results Data have been collected and are undergoing case-based analysis
The analysis included 62 schools comprising of 215 teachers in the that relies on: 1. Qualitative thematic analysis to capture clinician de-
intervention arm and 181 teachers in the control arm. The interven- scriptions of mandate implementation; 2. Cross-case analysis of quali-
tion had a positive effect on knowledge (0.30 [95% CI: 0.15 to 0.46]), tative and quantitative data to derive site-level indicators; 3.
environmental context and resources (0.50 [0.31 to 0.69]), social influ- Qualitative Comparative Analysis (QCA) to identify contextual factors
ences (0.18 [0.01 to 0.35]) but did not have an effect on beliefs about associated with mandate implementation outcomes. [3]
consequences (0.07 [-0.03 to 0.17]). All putative mediators were not Conclusions
associated with the primary outcome. Organizations implementing policy mandates for EBPs should con-
Conclusions sider how site-specific contextual drivers may interact with imple-
Although the implementation strategy caused meaningful improve- mentation efforts, affect mandate outcomes, and need to be
ments in scheduled minutes of physical activity, this effect was not addressed through targeted implementation strategies.
mediated by targeted mechanisms. Future research should explore Trial registration: ClinicalTrials.gov NCT02266134
the role of other potential mechanisms and evaluate system-level
mechanisms informed by an ecological framework. References
1. Scott K, Lewis CC. Using measurement-based care to enhance any treat-
ment. Cogn Behav Pract. 2015;22(1):49-59.
References
2. Lewis CC, Scott K, Marti CN, Marriott BR, Kroenke K, Putz JW, Mendel P,
1. Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, Hardeman W,
Rutkowski S. Implementing measurement-based care (iMBC) for depres-
Moore L, O’Cathain A, Tinati T, Wight D, Baird J. Process evaluation in complex
sion in community mental health: a dynamic cluster randomized trial
public health intervention studies: the need for guidance. J Epidemiol
study protocol. Implement Sci. 2015;10:127.
Community Health. 2014;68(2):101-2. doi:10.1136/jech-2013-202869.
3. Mendel P, Chen EK, Green HD, Armstrong C, Timbie JW, Kress AM,
2. Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy:
Friedberg MW, Kahn KL. Pathways to medical home recognition: a
strategies for combining interventions at multiple levels. J Natl Cancer
qualitative comparative analysis of the PCMH transformation process.
Inst – Monogr. 2012;(44):34-41. doi:10.1093/jncimonographs/lgs001.
Health Ser Res. 2018;53(4):2523-2546.
3. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC,
McHugh SM, Weiner BJ. Enhancing the impact of implementation
strategies in healthcare: a research agenda. Front Public Health. 2019;7(3). A51
doi:10.3389/fpubh.2019.00003. Understanding the implementation of evidence-informed policies
4. Nathan N, Wiggers J, Bauman AE, Rissel C, Searles A, Reeves P, and practices from a policy perspective: a critical interpretive
Oldmeadow C, Naylor PJ, Cradock AL, Sutherland R, Gillham K, Duggan B, synthesis
Chad S, McCarthy N, Pettett M, Jackson R, Reilly K, Herrmann V, Hope K, Heather L. Bullock, John N. Lavis, Michael G. Wilson, Gillian
Shoesmith A, Wolfenden L. A cluster randomised controlled trial of an Mulvale, Ashleigh Miatello
intervention to increase the implementation of school physical activity McMaster University, Hamilton, ON, Canada
policies and guidelines: study protocol for the physically active children Correspondence: Heather L. Bullock ([email protected])
in education (PACE) study. BMC Public Health. 2019;19(1):170. Implementation Science 2020, 15(Suppl 2):A51
doi:10.1186/s12889-019-6492-z.
5. Lee H, Herbert RD, Mcauley JH. Mediation Analysis. JAMA. 2019; Background
321(7):697-698. doi:10.1001/jama.2018.21973. The fields of implementation science and knowledge translation have
6. Imai K, Keele L, Tingley D. A general approach to causal mediation evolved somewhat independently from the field of policy implemen-
analysis. Psychol Methods. 2010;15(4):309-334. doi:10.1037/a0020761. tation research, despite calls for better integration [1]. As a result,
implementation theory and empirical work do not often reflect the training and technical assistance (TA) with a quality assurance measure
implementation experience from a policy lens nor benefit from the to develop a robust support system for implementation. Proactive TA is
scholarship in all three fields. This means policy makers, researchers delivered via weekly conference calls (~250 attendees/week) and a sup-
and practitioners may find it challenging to draw from theory that port email address (~250 emails/month). A SharePoint site which
adequately reflects their implementation efforts. houses a variety of tools developed for VA Risk ID is also utilized. The
Materials and Methods team also conducted a webinar series, offering training on the overall
We developed an integrated theoretical framework of the implementa- strategy and practice components, which was converted into VA online
tion process from a policy perspective by combining findings from learning system trainings. A fallout report was developed for quality as-
these fields using the critical interpretive synthesis method [2]. We surance, which provides information about patients who did not re-
began with the compass question: how is policy currently described in ceive indicated levels of the screening and evaluation process.
implementation theory and processes and what aspects of policy are Results
important for implementation success? We then searched 12 databases The above strategies allowed the implementation team to get real-time
as well as grey literature and supplemented these documents with feedback from the field, which was then communicated directly to VA
other sources to fill conceptual gaps. Using a grounded and interpret- policy makers on a weekly basis. As a result, major alterations have been
ive approach to analysis, we built the framework constructs and used made to the VA Risk ID implementation timeline and requirements.
our findings to consider improvements to existing theory. Conclusions
Results The presenters will discuss how VA Risk ID implementation serves as
A total of 7850 documents were retrieved and assessed for eligibility and a rich and useful example of how research, policy, implementation
34 additional documents were identified through other sources. Eighty- science and practice inform one another to result in the successful
two unique documents were ultimately included in the analysis. Our find- implementation of the largest suicide risk screening and evaluation
ings indicate that policy is described as: 1) the context; 2) a focusing lens; strategy in any United States healthcare system.
3) the innovation itself; 4) a lever of influence; 5) an enabler/facilitator or
barrier; or 6) an outcome. Policy actors were also identified as important References
participants or leaders of implementation. Our analysis led to the devel- 1. Luoma JB, Martin CE, Pearson JL. Contact with mental health and
opment of a two-part conceptual framework, including process and de- primary care providers before suicide: a review of the evidence. Am J
terminant components. We also used our findings to modify the Psychiatry. 2002;159:909–916.
Interactive Systems Framework for Dissemination and Implementation 2. Gairin I, House A, Owens D. Attendance at the accident and emergency
[3]. Finally, we provide an example of how the framework can be applied department in the year before suicide: retrospective study. Br J
using a policy implementation case from Ontario, Canada. Psychiatry. 2003;183:28–33.
Conclusions 3. Denneson LM, Kovas AE, Britton PC, Kaplan MS, McFarland BH, Dobscha
This framework begins to bridge the divide between disciplines and SK. Suicide risk documented during veterans last Veterans Affairs health
offers a new way of thinking about implementation processes at the care contacts prior to suicide. Suicide Life Threat Behav. 2016;46(3):363-
systems level. 374. doi:10.1111/sltb.12226.
4. The Joint Commission. Detecting and treating suicide ideation in all
References settings. Sentinel Event Alert. 2016;56:1-7.
1. Nilsen P, Stahl C, Roback K, Cairney P. Never the twain shall meet? A 5. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for
comparison of implementation science and policy implementation innovation support for implementing innovations with quality: tools, train-
research. Implement Sci. 2013;8:63. ing, technical assistance, and quality assurance/quality improvement. Am J
2. Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Comm Psychol. 2012;50(3-4):445-459. doi:10.1007/s10464-012-9509-7.
Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ. Conducting a
critical interpretive synthesis of the literature on access to healthcare by
vulnerable groups. BMC Med Res Methodol. 2006;6(1):35. A53
3. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, The role of implementation science in achieving health equity
Blachman M, Dunville R, Saul J. Bridging the gap between prevention Chair: Amanda Farley7, Discussant: Lisa Saldana8, Panelists: Allison
research and practice: the interactive systems framework for dissemination Metz1, Ana Baumann2, Leopoldo Cabassa2, Kimberly DuMont3, Beadsie
and implementation. Am J Community Psychol. 2008;41(3-4):171-81. Woo4, JD Smith5, Inger Burnett-Zeigler5, Juan Villamar5, Carlos
Gallo5, Hendricks Brown5, Moira McNulty6
1
University of North Carolina, Chapel Hill, Chapel Hill, NC, USA; 2Brown
A52 School, Washington University at St. Louis, St. Louis, MO, USA; 3William T.
Integrating research, policy, and practice to implement the largest Grant Foundation, New York, NY, USA; 4Annie E. Casey Foundation,
suicide risk identification strategy in a United States healthcare Baltimore, MD, USA; 5Northwestern University, Evanston, IL, USA;
6
system University of Chicago, Chicago, IL, USA; 7University of Birmingham,
Bridget Matarazzo, Nazanin Bahraini, Suzanne McGarity, Megan Birmingham, England, UK; 8Oregon Social Learning Center, Eugene, OR,
Harvey, Lisa Brenner USA
Rocky Mountain MIRECC for Suicide Prevention, Denver, CO, USA Correspondence: Allison Metz ([email protected])
Correspondence: Bridget Matarazzo ([email protected]) Implementation Science 2020, 15(Suppl 2):A53
Background
Background Inequities in healthcare are unfair differences between populations in the
Research suggests that a significant number of individuals who died by access, use, quality and outcomes of care. These inequities are persistent,
suicide were not identified as psychiatric patients nor were they receiving detrimental and costly. Implementation science has great potential to im-
mental health care; rather, they were often seen in primary care, ED or prove the health of communities and individuals that experience dispar-
other medical settings before their death [1-3]. In response to these find- ities. Equitable implementation occurs when strong equity components
ings and a Joint Commission Sentinel Event Alert [4], in October 2018, (including explicit attention to culture, history, values, and needs of the
the Department of Veterans Affairs (VA) launched the VA Suicide Risk community) are integrated into the principles and tools of implementa-
Identification Strategy (VA Risk ID), the largest population-based screen- tion science to facilitate quality implementation of effective programs for
ing and evaluation strategy in any United States healthcare system. Suc- a specific community or group of communities.
cessful implementation of VA Risk ID relies on collaboration between Materials and Methods
researchers, policy makers, and supervisors and providers within the field. This presentation includes two approaches to using implementation
Materials and Methods methods and frameworks to address healthcare inequities. The first
Consistent with the Evidence-Based System for Innovation Support approach uses Proctor et al. [1] framework to reframe five elements
Logic Model [5], the VA Risk ID implementation team combined tools, of implementation science to: 1) focus on reach from the very
beginning; 2) design and select interventions for vulnerable popula- collaborative care model for mood disorders across 57 community
tions with implementation in mind; 3) implement what works and practices in Michigan and Colorado.
develop implementation strategies that can help reduce inequities in Results
care; 4) develop the science of adaptations; and 5) use an equity lens In the first study, the rate of Re-Engage uptake (number of
for implementation outcomes. attempted patient contacts) was greater for enhanced REP sites com-
The second approach discusses three innovative implementation pared with standard REP sites (41% versus 31%, p=.01). An initial cost
method paradigms to improve scientific and health equity: 1) making analysis found that the additional time cost of Facilitation was 7.3
efficient use of existing data by applying epidemiologic and simula- hours per site, or ~$2500 per 6-month dose of Facilitation. In ADEPT,
tion modeling to understand what drives disparities and how they patients at sites receiving External Facilitation alone compared to
can be overcome; 2) designing new research studies that include, External+Internal Facilitation had improved SF-12 and mood symp-
but do not focus exclusively on populations experiencing disparities tom scores and higher odds of receiving collaborative care. The
in such areas as cardiovascular disease and co-occurring mental added costs of Internal Facilitation did not lead to greater implemen-
health conditions; and 3) research that focuses exclusively on popula- tation value, suggesting that REP plus External Facilitation was the
tions that have experienced high levels of disparities. most cost-effective combination of implementation strategies.
Results Conclusions
Conceptual approaches will initiate a much-needed dialogue on how to We discuss strengths and limitations of the economic evaluations
critically infuse an equity approach in implementation science to pro- performed in these studies. We also highlight how adaptive and
actively address healthcare inequities. These approaches raise numerous SMART designs are practical ways to assess the costs of implementa-
barriers for implementation research and how they can exacerbate dis- tion strategies and inform more efficient use of these strategies.
parities [2]. This work extends examples in behavioral health [3]. Trial registration: Current Controlled Trials ISRCTN21059161; Clinical-
Conclusions Trials.gov NCT02151331
Discussion will center on themes for taking action to ensure imple-
mentation science amplifies equity. Themes were identified through
structured facilitations with 21 researchers and include: employ strat- A55
egies and build structures that elevate equity concerns; shift funding Use of discrete choice experiments to inform stakeholder decision-
incentives to value practice expertise and questions; and promote ex- making about implementation
changes between researchers and community members. Ramzi Salloum1, Elizabeth Shenkman1, Stephanie Staras1, Jordan
Louviere2, David Chambers3
References
1
University of Florida, Gainesville, FL, USA; 2University of South Australia,
1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Adelaide, South Australia, Australia; 3National Cancer Institute, Bethesda,
Griffey R, Hensley M. Outcomes for implementation research: conceptual MD, USA
distinctions, measurement challenges, and research agenda. Adm Policy Correspondence: Ramzi Salloum ([email protected])
Ment Health. 2011;38(2):65-76. Implementation Science 2020, 15(Suppl 2):A55
2. Woodward EN, Matthieu MM, Uchendu US, Rogal S, Kirchner J.E. The
health equity implementation framework: proposal and preliminary study Background
of hepatitis C virus treatment. Implement Sci. 2019;14(1), 26. The discrete choice experiment (DCE) is a stated preference tech-
3. McNulty M, Smith JD, Villamar J, Burnett-Zeigler I, Vermeer W, Benbow N, nique from health economics for eliciting individual preferences over
Gallo C, Wilensky U, Hjorth A, Mustanki B, Schneider J. Implementation re- hypothetical alternative scenarios. This dynamic approach can be
search methodologies for achieving scientific equity and health equity. used to systematically measure the health preferences of various
Ethn Dis. 2019;29(Suppl 1):83-92. stakeholders – such as patients, providers, and administrators – and
thus engage those stakeholders’ perspectives in decisions about
investing time, money, and resources into implementation. The pur-
A54 pose of this presentation is to discuss the application of DCEs as a
Economic evaluation of implementation strategies: making the stakeholder engagement strategy to inform implementation of
business case for implementation science in the real world evidence-based interventions in health.
Amy M. Kilbourne1,2, Andria Eisman1, Daniel Eisenberg1 Materials and Methods
1
University of Michigan, Ann Arbor, MI, USA; 2Quality Enhancement This presentation will cover findings from a recent systematic review by
Research Initiative, U.S. Department of Veterans Affairs, Ann Arbor, MI, USA Salloum et al. [1] reporting on DCE applications in implementation re-
Correspondence: Amy M. Kilbourne ([email protected]) search. In addition, the presentation will include examples of DCE appli-
Implementation Science 2020, 15(Suppl 2):A54 cations from cancer prevention and control (e.g., tobacco control and
HPV vaccination) that vary by application type. The presentation will
Background discuss considerations for designing and conducting DCEs in imple-
Implementation strategies are methods used to help provider organiza- mentation research at each of the following stages: (1) identify and
tions deploy evidence-based practices. To date, few studies have com- characterize alternative scenarios; (2) experimental design to determine
pared the effectiveness of different implementation strategies, and rarely choices; (3) data collection; and (4) data analysis and interpretation.
have they assessed economic impact. Estimating costs of implementation Results
strategies – and comparing those costs to value generated – is crucial if DCE applications in implementation research can be categorized into
health care providers are to make informed decisions about investment four types: (1) characterizing demand for therapies and treatment tech-
in specific strategies to improve uptake of effective practices. nologies; (2) comparing implementation strategies; (3) prioritizing inter-
Materials/Methods ventions; and (4) incentivizing providers. An example of each
We describe two studies involving economic evaluations of the Repli- application type will be presented. A variety of stakeholders can be en-
cating Effective Programs (REP) implementation strategy. Both studies gaged using DCEs, including healthcare providers, patients, caregivers,
had adaptive designs in which sites that did not respond to 6 months and healthcare administrators. DCEs can be conducted across settings
of REP were randomized to receive additional implementation support and contexts, including clinical settings (inpatient and outpatient),
(i.e., Facilitation). First, Re-Engage was an adaptive implementation trial community-based settings, and at the policy/population level.
comparing REP alone to Enhanced REP (added External Facilitation) to Conclusions
enhance the uptake of a brief care management program for Veterans The use of DCEs to inform implementation of health interventions
with serious mental illness among a national cohort of 88 VA facilities. has been growing in recent years. As DCEs are more widely used in
Second, the Adaptive Implementation of Effective Programs Trial (ADEP health-related assessments, there is a wide range of applications for
T) was a cluster-randomized sequential multiple assignment random- them in the area of stakeholder engagement. Using DCEs can inform
ized trial (SMART) trial that compared REP only to REP adding External stakeholder decision making and support successful investment into
Facilitation or External+Internal Facilitation to improve the uptake of a implementation of health interventions.
Reference 3. Hovmand
1. Salloum PS. Community Based System Dynamics. New York, NY: Springer New York;
RG, Shenkman EA, Louviere JJ, Chambers DA. Application of discrete choice 2014.
experiments to enhance stakeholder engagement as a strategy for
advancing implementation: a systematic review. Implement Sci. 2017;12:140. A57
doi:10.1186/s13012-017-0675-8 Mixed-method approaches to strengthen economic and cost
research methods in implementation science
A56 Alex R. Dopp1, Peter Mundey2, Lana O. Beasley3, Jane F. Silovsky2, Daniel
Modeling to learn: conserving staff time when comparing Eisenberg4
1
implementation alternatives via simulation RAND Corporation, Santa Monica, CA, USA; 2University of Oklahoma
Lindsey Zimmerman1, David Lounsbury2, Tom Rust3, Craig Rosen1, Rachel Health Sciences Center, Oklahoma City, OK, USA; 3Oklahoma State
Kimerling1, Jodie Trafton4, Steven Lindley5, Andrew Holbrook1, Stacey Park1, University, Stillwater, OK, USA; 4University of Michigan, Ann Arbor, MI, USA
Jane Branscomb6, Debra Kibbe6, James Rollins7, Savet Hong1 Correspondence: Alex R. Dopp ([email protected])
1
National Center for PTSD, Menlo Park, CA, USA; 2Albert Einstein College Implementation Science 2020, 15(Suppl 2):A57
of Medicine, Bronx, NY, USA; 3Center for Healthcare Transformation,
Boston, MA, USA; 4Program Evaluation Resource Center, Palo Alto, CA, Background
USA; 5Veterans Affairs Palo Alto Health Care System, Palo Alto, CA, USA; Guidance on the costs and economic impacts of implementing
6
Georgia Health Police Center, Georgia State University, Atlanta, GA, USA; evidence-based practices is critical to informing investments in imple-
7
Takouba Security LLC, Seattle, WA, USA mentation efforts. However, the results of traditional methods – such as
Correspondence: Lindsey Zimmerman ([email protected]) economic evaluations, discrete choice modeling, and participatory sys-
Implementation Science 2020, 15(Suppl 2):A56 tem dynamics modeling – are limited by a remaining “qualitative re-
sidual” of contextual information and stakeholders’ perspectives. This
Background residual, which is particularly prevalent in implementation research,
For over 15 years, VA has implemented national dissemination efforts cannot be fully captured by the quantitatively-based analyses and
to train providers in evidence-based addiction and mental health models used in these methods. The emergence of qualitative methods
practices (EBPs). VA mandates EBPs and supports their implementa- for studying economics and costs offers a promising solution.
tion with substantial investment in infrastructure to support quality Materials and Methods
improvement (e.g., incentivized VA-wide quality measures), yet only We recommend that researchers maximize their contributions related
3-28% of the patient population receives the highest quality care. To to economics and costs within implementation science by embracing
improve EBP access, the National Center for PTSD developed a a mixed-methods research agenda that merges traditional quantita-
simulation-learning program designed to help frontline teams iden- tive approaches with innovative, contextually grounded qualitative
tify locally calibrated EBP improvement strategies [1]. We piloted par- methods. Such studies are exceedingly rare at present.
ticipatory system dynamics (PSD) for improving EBP reach based on Results
its effectiveness in business and engineering [2-3]. To assist implementation scientists in making use of mixed methods in
Materials and Methods this research context, we will present an adapted taxonomy that de-
PSD synthesizes engagement principles and state of the science tech- scribes the structure and function mixed-method studies relevant to eco-
nologies for understanding and changing systems. As a systems sci- nomic and cost research. We will then illustrate the application of mixed
ence method, PSD demonstrates how causal system properties vary as methods in exemplar studies that used economic evaluation, discrete
a function of local resources (e.g., financial, personnel). Over the last choice modeling, or system dynamics methods to study implementation.
four years, in partnership with patient, provider, and policy-maker The examples presented will emphasize the breadth of qualitative
stakeholders, we co-developed a PSD program entitled Modeling to methods for data collection (e.g., interviews, focus groups, site visits, re-
Learn (MTL). MTL supports multidisciplinary frontline teams of providers view of records, ethnography) and analysis (e.g., content analysis, the-
to address high priority areas (e.g., Suicide Prevention, Access to Care, matic analysis, grounded theory, case studies) that can be incorporated
Opioid Misuse, PTSD) by conserving local staff time in simulation learn- into studies of implementation costs and economics. Finally, we will re-
ing models that evaluate and compare implementation scenarios. view reporting guidelines for these methods (e.g., Consolidated Health
Results Economic Evaluation Reporting Standards) with an emphasis on how to
MTL helped two pilot clinics increase EBP reach with existing local staff re- incorporate mixed methods into existing guidelines.
sources. Preliminary statistical process control analyses indicate pilot clinics Conclusions
demonstrated a three standard deviation increase in EBP reach and main- By incorporating qualitative methods, implementation researchers
tained improvement for 12 and 8 months, respectively. We will present ex- can enrich their research on economics and costs with detailed,
ample simulation experiments for different VA clinics, highlighting how context-specific information to tell the full story of the economic im-
simulation helps staff optimize local staff resources to meet patients’ needs. pacts of implementation. We will end by providing suggestions for
Conclusions building a research agenda in mixed-method economic evaluation,
Most cost analysis in implementation research focuses on the costs of EBP along with more resources and training to support investigators who
adoption. But, VA-adopted EBPs and VA budgets facilitate/constrain the wish to answer our call to action.
resources needed to expand reach. We posit that MTL created greater
consensus about change (front-end optimization) and guided more effect- A58
ive investment decisions within local system resources (back-end Developing the adaptation-impact model and translating it for use
optimization). Additional study of the MTL national rollout is underway: a in practice
multisite cluster randomized trial will test the superiority of MTL over M. Alexis Kirk1, Julia E. Moore2, Byron J. Powell3, Sarah Birken1
1
audit-and-feedback for its effectiveness improving the reach of EBPs. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; 2The
Center for Implementation, Toronto, ON, Canada; 3Washington
References University in St. Louis, St. Louis, MO, USA
1. Zimmerman Correspondence: M. Alexis Kirk ([email protected])
L, Lounsbury DW, Rosen CS, Kimerling R, Trafton JA, Lindley SE. Participatory Implementation Science 2020, 15(Suppl 2):A58
system dynamics modeling: increasing stakeholder engagement and
precision to improve implementation planning in systems. Adm Policy Ment Background
Health. 2016;43(6):1-16. doi:10.1007/s10488-016-0754-1 Implementation science is shifting from qualifying adaptations as
2. Sterman good or bad towards understanding nuances of adaptations and
JD. Learning from evidence in a complex world. Am J Public Health. their impact. Existing adaptation classification frameworks are largely
2006;96:505–514. doi:10.2105/AJPH.2005.066043 descriptive (e.g., who made the adaptation) and geared towards
researchers. They do not help practitioners in decision-making around program with youth 10-14 years old and their parents) implemented
adaptations – is an adaptation likely to have negative impacts? Should in natural contexts across Washington State. Our previous work ap-
it be pursued? Moreover, they lack constructs to consider potentially plied and extended two multidimensional coding systems [1-2] to
disparate impact on intervention and implementation outcomes (e.g., 154 implementer-reported adaptations of SFP [3]. Based on these re-
whether an adaptation might improve implementation outcomes but sults, we designed a quantitative measure to track the extent to
weaken intervention outcomes). which implementers modified, added, and deleted program content/
Materials and Methods processes; the most common types and reasons for adaptations; and
We consolidated two adaptation frameworks [1-2] and one the extent to which adaptations proactive or reactive.
intervention-implementation outcome framework [3]. We reviewed Results
each framework to refine constructs and group them into domains. We Preliminary results from 28 SFP implementations show that over half re-
then coded qualitative descriptions of 14 adaptations from an existing port modifying the program content/processes “a little”, but 60% report
intervention being adapted to a new context to test fit of our frame- “not at all” when asked about addition or deletion of content/processes.
work. To bridge the research-practice gap, we then developed guid- The most commonly reported modifications were made to games and
ance to help practitioners in the field apply this framework to activities/icebreakers, and the most commonly reported reason was lack
adaptation efforts. of time/competing demands on time. “Need for a more culturally appro-
Results priate program” was reported by 24% despite 39% of implementations
Our framework has 3 domains and our applied guidance for each do- being delivered in Spanish or bilingually. Planning in advance “some” or
main is as follows: “a lot” for adaptations was reported by 37%, whereas 15% reported
Criteria for making adaptations: useful in considering whether adapta- “some” or “a lot” for making reactive adaptations.
tions will have a positive or negative impact to help decide whether to Conclusions
move forward with the adaptation. Systematic adaptations with a posi- These results help describe the adaptations being made and how/
tive valence are more likely to have a positive impact. why they are being made. Future analyses will examine links be-
Impact of adaptations: useful in considering adaptations’ impact on tween these dimensions of adaptation and participant engagement,
intervention outcomes (intervention effectiveness) and implementa- which increasingly is shown to be a critical mediator/moderator be-
tion outcomes (acceptability, cost, etc.). to help anticipate and miti- tween adaptations and program outcomes. Ultimately, this work will
gate negative consequences of adaptations (e.g., plan for help develop a theoretically- and empirically-grounded tool to track
implementation of an adaptation that might improve intervention ef- real-world adaptations and their impacts, in turn enhancing program
fectiveness but decrease acceptability). implementation guidance on how to strike the right balance be-
Adaptation Typology: classifies adaptation attributes (e.g., type, na- tween program fidelity and adaptation.
ture of adaptation), providing consistency in reporting
Conclusions References
Our guidance takes a consolidated framework developed for re- 1. Hill LG, Maucione K, Hood BK. A focused approach to assessing program
search and translates it to be helpful to practitioners. Our guidance fidelity. Prev Sci. 2007; 8(1):25-34.
helps practitioners “back-up” and re-think adaptations suspected to 2. Moore JE, Bumbarger BK, Cooper BR. Examining adaptations of evidence-
have negative impacts and helps practitioners think through “ripple based programs in natural contexts. J Prim Prev. 2013;34(3):147-161.
effects” and tradeoffs of adaptations to plan accordingly (e.g., put in 3. Cooper BR, Shrestha G, Hyman L, Hill LG. Adaptations in a community-
place an implementation strategy to monitor/boost fidelity if an based family intervention: replication of two coding schemes. J Prim
adaptation is suspect to have positive impacts on acceptability, but Prev. 2016;37:33-52.
negative impacts on fidelity).
References A60
1. Moore JE, Bumbarger BK, Cooper BR. Examining adaptations of evidence- Rapid adaptation: making adaptations work for real world systems,
based programs in natural contexts. J Prim Prev. 2013:34(3):147-161. services, and science
2. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework Sarah Cusworth Walker1, Michael Graham-Squire2
1
and coding system for modifications and adaptations of evidence-based University of Washington, Seattle, WA, USA; 2Neighborhood House,
interventions. Implement Sci 2013:8(1):65. Seattle, WA, USA
3. Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Correspondence: Sarah Cusworth Walker ([email protected])
Griffey R, Hensley M. Outcomes for implementation research: conceptual Implementation Science 2020, 15(Suppl 2):A60
distinctions, measurement challenges, and research agenda. Adm Policy
Ment Health. 2011;38(2):65-76. Background
To reach scale, public health agencies and researchers will need to
partner in new ways to meet the prevention needs of diverse and
A59 dynamic communities. Building from concepts proposed in the Dy-
Striking the right balance: tracking adaptations to community- namic Adaptation Framework [1], common elements [2], and adapta-
based prevention programs to enhance guidance to implementers tion models [3], we present preliminary results from a codesign
Brittany Cooper, Garrett Jenkins, AnaMaria Diaz Martinez process developed to provide tailored, rapid guidance for delivering
Washington State University, Pullman, WA, USA more culturally congruent prevention services. We present data from
Correspondence: Brittany Cooper ([email protected]) a six-month codesign demonstration project in Seattle, WA between
Implementation Science 2020, 15(Suppl 2):A59 the University of Washington and a local social services agency with
over 100 years of delivering social welfare and prevention services.
Background The model is proceeding in four stages with distinct data capture at
The adoption of effective programs is insufficient for achieving posi- each phase: 1) Identify areas of strain; 2) Derive Core Elements; 3) De-
tive youth and family outcomes community-based organizations velop guidance; 4) Test acceptability.
seek; high quality implementation is also critical. However, making Materials and Methods
decisions about dosage, content, and structure of an evidence-based We present data on the need for adaptation and the feasibility of the
program as it was originally designed (i.e., fidelity) while adapting to codesign model from the 21 facilitator respondents and the feasibil-
local contexts is challenging and complex, especially under resource ity of the codesign process from community and research partici-
strain. Implementers are often left to make these decisions without pants. Strain was assessed through surveys distributed to 21 GGC
much empirically-based guidance. facilitators. The surveys asked respondents to score each activity in
Materials and Methods the curriculum for ease of delivery and need for adaptation. Qualita-
We used data from an ongoing, large-scale evaluation of Strengthen- tive responses were coded using the theoretical framework of struc-
ing Families Program (SFP; a 7-week substance use prevention ture and content adaptations [3].
Results strategies to enhance motivation whereas over-adapting adopters

The fidelity of delivery to specific activities within GGC varied signifi- may need support for goal setting and/or appropriate adaptations.
cantly, with some activities rarely delivered according to manualized Trial registration: ClinicalTrials.gov NCT03075085
instructions and other always or almost always delivered as written
(range of 2.05-4.48 on a 5-point scale with 5 keyed to “always”). Ac- A62
tivities rarely delivered as written were more likely to be experiential Development of an instrument for evaluating implementation
activities (e.g., group sculpture or scavenger hunt). Activities deliv- efforts and benchmarking regarding person centered care
ered as intended but rated as poor on cultural responsivity included Helena Fridberg1, Lars Wallin1,2, Catarina Wallengren2, Henrietta
videos and scripted content delivered by facilitators. Forsman1, Anders Kottorp3, Malin Tistad1
1
Conclusions Dalarna University, Falun, Sweden; 2Gothenburg University, Gothenburg,
A number of structural and content adaptations needs were indi- Sweden; 3Malmö University, Malmö, Sweden
cated, including the need to address divergent cultural views about Correspondence: Helena Fridberg ([email protected])
some core assumptions of the GGC model. Implementation Science 2020, 15(Suppl 2):A62
References Background
1. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky Policy makers across Sweden are pushing implementation of person
JP, Hecht DB, Chaffin MJ. Dynamic adaptation process to implement an centered care (PCC) in health care settings as a way to promote high
evidence-based child maltreatment intervention. Implement Sci. 2012;7(1):32-40. quality health care across the country [1]. However, there is a lack of
2. Lyon AR, Lau AS, McCauley E, Vander Stoep A, Chorpita BF. A case for valid and reliable instruments to measure and compare patients’ expe-
modular design: implications for implementing evidence-based interven- riences of PCC across health care settings and patient groups. As part
tions with culturally diverse youth. Prof Psychol Res Pract. 2014;45(1):57-66. of a research project aimed at investigating implementation strategies
3. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework of PCC at a regional level, we set out to develop a generic instrument
and coding system for modifications and adaptations of evidence-based to measure patients’ experiences of PCC as an outcome of the imple-
interventions. Implement Sci. 2013;8:65. mentation efforts [2, 3]. In this project, we collaborate with the Swedish
association of local authorities and regions (SALAR) who are responsible
for the administration of the largest patient survey in Sweden. When
A61 complete, the instrument will be available to healthcare regions for
What fidelity data don’t say: types of adopters and resisters in an evaluating implementation and development of the PCC approach.
implementation trial in early care and education classrooms Materials and Methods
Taren Swindle1, Julie Rutledge2, Geoffrey Curran1 A mixed methods design is used, entailing the following phases: con-
1
University of Arkansas for Medical Sciences, Little Rock, AR, USA; struction of a preliminary questionnaire based on questions from SALA
2
Louisiana Tech University, Ruston, LA, USA R’s existing item pool, content validation of items via experts (patients,
Correspondence: Taren Swindle ([email protected]) healthcare practitioners and researchers) in a Delphi study, cognitive
Implementation Science 2020, 15(Suppl 2):A61 interviewing with patients, data collection and psychometric evaluation
through Rasch analyses. The last two phases are repeated in two
Background rounds. Finally, the instrument will be handed over to SALAR who will
Together, We Inspire Smart Eating (WISE) is a nutrition promotion be responsible for translation into seven languages, design of web and
intervention designed for delivery by early care and education paper-based surveys, and procedures for recruitment of patients.
teachers. Data from a prior implementation of WISE showed subopti- Results
mal fidelity to its four, key evidence-based practices (i.e., hands-on Data collection for the first psychometric evaluation will end by mid-
exposure to fruits and vegetables (FV), use of mascot to promote FV, April. Preliminary results from the first phase of Rasch analyses are
positive feeding practices, and role modeling). We are currently test- expected by the end of May. The results from the second phase are
ing strategies to support uptake of WISE. This study presents prelim- expected by the end of August. The finalized instrument will be
inary findings on the behaviors observed by teachers. launched by SALAR in the beginning of 2020.
Materials and Methods Conclusions
A two-arm implementation trial is ongoing in 40 classrooms to implement We set out to develop a robust and psychometrically sound instru-
WISE with a focus on its evidence-based practices. All classrooms are ob- ment to measure patients’ perceptions on PCC. The collaboration
served on a quarterly basis for fidelity by data collectors trained to 85% reli- with SALAR has been fruitful and will greatly expand survey oppor-
ability. Classrooms in the treatment condition (i.e., enhanced support, N = 17) tunities. We expect the instrument to be widely used for evaluating
receive targeted implementation support based on their observed fidelity, implementation efforts and benchmarking regarding PCC.
which included facilitation. After the first and second quarter of the school
year, 5 classrooms from the pool of poorest fidelity performers in the en- References
hanced condition were randomly selected for semi-structured interviews. To 1. Ekman I, Swedberg K, Taft C, Lindseth A, Norberg A, Brink E, Carlsson J,
date, the second quarter of data collection, 6 months of enhanced support, Dahlin-Ivanoff S, Johansson IL, Kjellgren K, Lidén E, Öhlén J, Olsson LE,
and both rounds of interviews are complete. Facilitators derived types of Rosén H, Rydmark M, Sunnerhagen KS. Person-centered care–ready for
adopters and resisters based on analysis of observational and interview data. prime time. Eur J Cardiovasc Nurs. 2011;10(4):248-251.
Results 2. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in
Four types of adopters and resisters were identified: enthusiastic implementation science. Implement Sci. 2014;9:118.
adopters (35%), over-adapting adopters (24%), soft resisters (17%), and 3. Moore L. Britten N. Lydahl D, Naldemirci Ö, Elam M, Wolf A. Barriers and
hard resisters (24%). Enthusiastic adopters exhibited positive attitudes facilitators to the implementation of person centred care in different
towards WISE and moderate to strong fidelity. Over-adapting adopters, healthcare contexts. Scand J Caring Sci. 2017;31:662–673
while exhibiting positive attitudes toward WISE, made fidelity-
inconsistent adaptations that were potentially detrimental (i.e., using
mascot to shame children). Soft resisters demonstrated poor to moder- A63
ate fidelity and showed lack of interest in adopting WISE or receiving Implementation practice track: where the rubber meets the road–
facilitation support. Hard resisters were vocal about their complaints in novel applications and adaptations of implementation tools and
adopting WISE and/or noticeably against receipt of facilitation support; strategies in real world settings
most, but not all, hard resisters had poor fidelity. Robert Franks1, Jonathan Scaccia2
1
Conclusions Judge Baker Children’s Center/Harvard, Boston, MA, USA; 2Wandersman
There are nuances in low fidelity not captured by quantitative scores. Center, Philadelphia, PA, USA
Different types of resisters may be best served with different imple- Correspondence: Robert Franks ([email protected])
mentation strategies. For example, hard resisters may benefit from Implementation Science 2020, 15(Suppl 2):A63
Background implementation activity was observed, reflective of funding pres-

There are many implementation and dissemination frameworks to help sures, putting implementation quality at risk. The programs were
to bridge the gap between research and practice. As implementation generally well-accepted, perceived as high-quality and a good fit.
science has grown, we must avoid the paradoxical gap between imple- While most agency staff ‘believed in’ the programs, perceived appro-
mentation research and implementation practice. This presentation will priateness was compromised by the lack of adaptability for Aborigi-
describe the collaboration between researchers and practitioners in the nal and Torres Strait Islander communities. Threats to feasibility
development and application of organizational readiness and the con- included high demands on practitioners and lack of Australian-based
tinued use of the R=MC2 heuristic in research, evaluation, and practice. implementation support (trainers, consultants). It was too early for
Materials and Methods valid fidelity assessments.
We will first briefly describe the development of a new definition and Conclusions
conceptualization of Organizational Readiness for understanding and Policy-makers should afford agencies more time/resources to incorp-
facilitating the implementation of innovations into new settings. De- orate initiatives into ‘business as usual’. Ongoing monitoring of im-
rived from work in the Interactive Systems Framework, the R=MC2 plementation outcomes is highly recommended to facilitate data-
model proposes that readiness is not a singular, static condition, but ra- driven decisions about when to start impact evaluation (i.e. when
ther a collection of dynamic constructs that includes motivation, gen- sustainability is achieved, and fidelity has been demonstrated).
eral capacity, and innovation-specific capacity. Readiness can be used
to monitor and facilitate implementation over time. The Wandersman References
Center developed a measure (the Readiness Diagnostic Tool) with good 1. Chamberlain P, Brown CH, Saldana L. Observational measure of
initial psychometric validity to help gather comprehensive information implementation progress in community based settings: the stages of
about facilitators of a change effort. implementation completion (SIC). Implement Sci. 2011;6(1):116
Results 2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC.
A goal in the development of the RDT was that it could be used in Fostering implementation of health services research findings into
multiple settings and for different interventions. Therefore, in this practice: a consolidated framework for advancing implementation
joint presentation between a researcher and a practitioner, we will science. Implement Sci. 2009;4(1):50.
also briefly describe the adaptation and design process for using the 3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
RDT in distinct settings and a new study to rigorously validate the MM, Proctor EK, Kirchner JE. A refined compilation of implementation
measure. The potential to adapt the readiness measure is key for the strategies: results from the Expert Recommendations for Implementing
application of implementation science constructs in real-world condi- Change (ERIC) project. Implement Sci. 2015;10:21.
tions. We will talk about how practitioners negotiate tradeoffs in
making implementation constructs more specialized and application
to their specific setting. Using the RDT as an example, we will share A65
how we collaboratively worked through these adaptation issues. Implementation science for depression interventions in low- and
Conclusions middle-income countries: a systematic review
The adaptation process and lessons learned were bidirectional. We Bradley Wagenaar1, Wilson Hammett1, Courtney Jackson1, Dana
will discuss how applied findings in real world settings informed Atkins1, Jennifer Belus2, Christopher Kemp1
1
changes in the underlying theory and measure, specifically about University of Washington, Seattle, WA, USA; 2University of Maryland,
what and how the constructs were useful. This presentation will College Park, MD, USA
highlight how theory and practice can inform one another in an it- Correspondence: Bradley Wagenaar ([email protected])
erative manner to help to move the science forward while promoting Implementation Science 2020, 15(Suppl 2):A65
a deeper understanding of implementation facilitators.
Background
Interventions to treat depression are demonstrating effectiveness across
A64 a range of low-resource settings globally. Significant investments are be-
How can implementation quality be evaluated? an example from a ing made to decrease the gap between this evidence and its application
pilot initiative in Australian child and family services at scale. Our objectives were to systematically review implementation re-
Correspondence: Vanessa Rose ([email protected]) search targeting depression interventions in low- and middle-income
Centre for Evidence & Implementation, Sydney, Australia countries (LMICs) and critically assess coverage and scientific gaps.
PubMed, CINAHL, PsycINFO, and EMBASE were searched for evaluations of
Background depression interventions in LMICs reporting at least one implementation
High-quality program implementation is a pre-condition to program ef- outcome. Study- and intervention-level characteristics were abstracted.
fectiveness. However, evaluation of the implementation process is rare, Results
resulting in uncertainty around interpretation of impact evaluations with A total of 7,034 studies were screened, 589 were assessed for eligibility,
null effects (i.e. was the program ineffective, or implemented poorly?). and 59 studies published between 2003 and 2017 met inclusion criteria.
We report on an implementation evaluation of the Victorian Govern- Most studies were conducted in Sub-Saharan Africa (n=24; 40.7%),
ment’s pilot of five manualized therapeutic programs for vulnerable fam- followed by South Asia (n=17; 28.8%), and Latin America and the Carib-
ilies (four developed in the USA) across seven service provider agencies; bean (n=12; 20.3%). The majority of studies (n=41; 69.5%) reported out-
the first evaluation of this nature and scope in Australia. The aim was to comes for a depression intervention that was implemented at the pilot/
provide an indication of the comprehensiveness, pace and quality of pro- research phase. The majority of studies (n=35; 59.3%) focused on depres-
gram implementation to inform government decisions about if/ how sive interventions delivered at the facility level, with 22 (37.3%) delivered
such programs should be funded, implemented, supported and scaled. in the community. Thirty studies (50.9%) utilized non-specialized health-
Materials and Methods care workers as the implementing agent. Primary depression intervention
A real-world mixed-methods observational study design was used. modalities were individual psychotherapy (n=20; 33.9%) and multicom-
The Stages of Implementation Completion checklist assessed imple- ponent interventions (n=20; 33.9%). Only 19 studies (32.2%) tested an im-
mentation pace and comprehensiveness [1]. Theory-based structured plementation strategy, with the most common implementation strategy
interviews were conducted with agency staff (N=29) to explore pro- being revising professional roles (n=8; 42.1%). The most common imple-
gram appropriateness, acceptability and feasibility [2]. Implementa- mentation outcomes reported were acceptability (n=39; 66.1%), followed
tion strategies were explored with manualized program purveyors by feasibility (n=22; 37.3%), and fidelity (n=16; 27.1%). No study reported
[3]. Fidelity data were extracted from agency databases. penetration, and only 3 (5.1%) reported adoption or sustainability.
Results Conclusions
Most agencies (n=6) were still in early implementation, having not Implementation research for depressive interventions in LMICs has
yet achieved sustainability. Highly-concentrated and overlapping focused largely on early-stage implementation outcomes. Most
studies have the primary aim of testing evidence-based interventions 3. Kohonen T. Self-Organizing Maps. 3rd edition. Berlin: Springer; 2000.
under pilot researcher-controlled implementation. Future implemen-
tation research could prioritize the development and testing of im- A67
plementation strategies to promote delivery of evidence-based Pragmatic measures for implementation research: development of
depression interventions in routine care. This would include in- the Psychometric and Pragmatic Evidence Rating Scale (PAPERS)
creased consideration of contextual factors, as well as later-stage im- Cameo F. Stanick1, Heather M. Halko2, Elspeth A. Nolen3, Byron J.
plementation outcomes such as cost, penetration, and sustainability. Powell4, Caitlin N. Dorsey5, Kayne D. Mettert5, Bryan J. Weiner3, Melanie
Certain regions, such as Middle East and North Africa, East Asia and Barwick6, Luke Wolfenden7, Laura J. Damschroder8, Cara C. Lewis5
Pacific, and Europe and Central Asia could be prioritized for invest- 1
Hathaway-Sycamores Child and Family Services, Pasadena, CA, USA;
ments given the paucity of research. 2
University of Montana, Missoula, MT, USA; 3University of Washington,
Seattle, WA, USA; 4Brown School, Washington University in St. Louis, St.
A66 Louis, MO, USA; 5Kaiser Permanente Washington Health Research
Predicting quality improvement sustainability with artificial neural Institute, Seattle, WA, USA; 6Hospital for Sick Children and University of
networks Toronto, Toronto, ON, USA; 7The University of Newcastle, Newcastle,
Tim Rappon1, Erica Bridge2, Alyssa Indar2, Whitney Berta2 Australia; 8VA Ann Arbor Healthcare System, Center for Clinical
1
University of Toronto, Toronto, ON, Canada; 2University of Toronto, Management Research (CCMR), Ann Arbor, MI, USA
Institute of Health Policy, Management and Evaluation, Toronto, ON, Correspondence: Cameo F. Stanick ([email protected])
Canada Implementation Science 2020, 15(Suppl 2):A67
Correspondence: Tim Rappon ([email protected])
The incorporation and use of reliable, valid measures in implementa-
Background tion practice, outside of research, will remain limited if measures are
Quality Improvement (QI) initiatives are proposed as key vehicles to not pragmatic. Previous research has identified the need for pragmatic
shift our health system from disease-focused, episodic care to com- measures, though the pragmatic properties identified were developed
prehensive care for older adults living with chronic conditions. With using only expert opinion and literature review. Our team carried out
the proportion of older Canadians set to double over the next two four studies with the goal of developing a stakeholder-driven prag-
decades, the need for QI is pressing, yet there is a dearth of research matic rating criteria for implementation measures. We previously pub-
on whether QI yields long-term changes in practice (sustainment) or lished Studies 1 (populating the dimensions of the pragmatic construct
benefits (sustainability) for older patients and residents [1]. Our study via a literature review + stakeholder interviews) and 2 (clarifying the in-
responds to Proctor et al.’s (2015) call to “test theories, frameworks ternal structure via concept mapping) that yielded 47 terms and
and models for their ability to explain and predict sustainability” [2]. phrases across four categories: Useful, Compatible, Acceptable, and
Materials and Methods Easy that were culled to 17 terms. This study presents the results of
We employed a Kohonen self-organizing map (SOM) [3] (an artificial Studies 3 and 4: a Delphi to ascertain stakeholder-prioritized dimen-
neural network) to identify factors associated with sustainment and sions and a pilot study applying the dimensions as rating criteria.
sustainability for QI interventions. To create our training dataset, we Materials and Methods
searched Medline, PsychINFO and CINAHL for articles which reported Stakeholders (practitioners and implementation intermediaries; N=
on the long-term (1+ years post-implementation) sustainability of QI 26) participated in an online modified Delphi and rated the relevance
programs targeted to older adults. After screening 3127 abstracts, 54 of 17 terms and phrases to the pragmatic construct. The investigative
articles were selected. Two coders independently extracted study team pruned the list further and developed anchors for the prag-
characteristics, including organizational & clinical context, implemen- matic properties based on all available data sources (i.e., stakeholder
tation & post-implementation strategies, and adaptations. We per- interviews, literature review, concept mapping ratings, Delphi rat-
formed leave-one-out cross-validation using the extracted variables ings). The final criteria were piloted with 60 existing implementation
to assess the sensitivity and specificity of the self-organizing map’s measures utilizing both empirical and grey literature.
predictions of sustainment and sustainability. Results
Results The Delphi methodology confirmed the importance of all identified prag-
The SOM achieved 38% sensitivity and 73% specificity for sustainment matic measure properties, but provided little guidance on relative import-
and 89% sensitivity and 42% specificity with respect to sustainability. The ance. Following the Delphi, investigators removed/combined 6 more terms
diagnostic odds ratio was not significant for sustainment but was 5.2 (sta- to obtain the final set of 11 criteria across four categories (Useful, Acceptable,
tistically significant p<0.05) for sustainability. We also estimated the rela- Easy, Compatible) and assigned a 6-point rating system to each criterion. Ap-
tive contribution of different determinants to the prediction of plication of the final rating criteria demonstrated sufficient variability across
sustainment and sustainability by observing how their omission affected items; the grey literature did not add critical information.
the predictive power of the SOM. Clinical targets, adaptations, post- Conclusions
implementation (sustainability) strategies, and organizational context This work produced the first stakeholder-driven rating criteria by which
were all significant predictors of sustainability; however, a larger training measures can be judged to be pragmatic. The Psychometric and Prag-
set would be required for a predictive model of sustainment and to matic Evidence Rating Scale (PAPERS) was developed by combining
generalize the SOM beyond QI initiatives in health care for older adults. these pragmatic criteria with psychometric rating criteria from our pre-
Conclusions vious work. Use of PAPERS can inform development of new implemen-
Our study presents a novel method for investigating relationships be- tation measures and to assess quality of existing measures.
tween (post-)implementation factors and sustainability, which could
be extended (with a larger training dataset) to produce predictions A68
of—and tailored recommendations for—intervention sustainment Psychometric and pragmatic evaluation of measures of readiness
and sustainability. for implementation
Bryan J. Weiner1, Caitlin N. Dorsey2, Kayne D. Mettert2, Cara C. Lewis2
1
References University of Washington, Seattle, WA, USA; 2Kaiser Permanente
1. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based in- Washington Health Research Institute, Seattle, WA, USA
terventions and practices in public health and health care. Annu Rev Public Correspondence: Bryan J. Weiner ([email protected])
Health. 2018;39(1):55-76. doi:10.1146/annurev-publhealth-040617-014731. Implementation Science 2020, 15(Suppl 2):A68
2. Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, Padek
M. Sustainability of evidence-based healthcare: Research agenda, meth- Background
odological advances, and infrastructure support. Implement Sci. Systematic measure reviews can facilitate advances in implementa-
2015;10(88):1-13. tion research and practice by locating reliable, valid, pragmatic
measures; identifying promising measures needing refinement and relative priority, organizational incentives and reward, goals and
testing; and highlighting measurement gaps. Sponsored by Society feedback, and learning climate.
for Implementation Research Collaboration (SIRC), with funding from Materials and Methods
the National Institute of Mental Health (NIMH), this review identifies This systematic review was conducted as a part of a larger study,
and evaluates the psychometric and pragmatic properties of mea- and the full protocol is published elsewhere [1]. The review pro-
sures of readiness for implementation and its sub-constructs as delin- ceeded in three phases. Phase I, data collection, involved search
eated in the Consolidated Framework for Implementation Research: string generation, title and abstract screening, full text review, con-
leadership engagement, available resources, and access to know- struct assignment, and measure forward searches. Phase II, data ex-
ledge and information. traction, involved coding relevant psychometric and pragmatic
Materials and Methods information. Phase III, data analysis, involved two trained specialists
The systematic review methodology is described fully elsewhere [1]. independently rating each measures’ psychometric properties. Fre-
Consistent with SIRC’s mission and NIMH’s priorities, the review fo- quencies and central tendencies summarized information availability
cused on measures used in mental or behavioral healthcare. The re- and psychometric ratings.
view proceeded in three phases. Phase I, data collection, involved Results
search string generation, title and abstract screening, full text review, Searches identified 65 measures or subscales assessing molar
construct assignment, and measure forward searches. Phase II, data organizational culture or climate, 6 measures of implementation cli-
extraction, involved coding relevant psychometric and pragmatic in- mate, 4 of which were subscales of other measures; and a number of
formation. Phase III, data analysis, involved two trained specialists in- other subscales for constructs related to the CFIR’s conceptualization
dependently rating each measure using PAPERS (Psychometric And of implementation climate, including: 2 assessing tension for change;
Pragmatic Evidence Rating Scale) [1]. Frequencies and central ten- 6 assessing compatibility; 2 assessing relative priority; 3 assessing
dencies summarized information availability and PAPERS ratings. organizational incentives and rewards; 3 assessing goals and feed-
Results back; and 2 assessing learning climate. Information about internal
Searches identified nine measures of readiness for implementation, consistency and norms was available for most measures. Information
24 measures of leadership engagement, 17 measures of available re- about other psychometric properties was often not available. Ratings
sources, and 6 measures of access to knowledge and information. In- for internal consistency were most often “adequate” or “good.” Rat-
formation about internal consistency was available for most ings for other psychometric properties were typically lower.
measures. Information about other psychometric properties was Conclusions
often not available. Ratings for internal consistency were “adequate” This review suggests some promising measures of organizational cul-
or “good.” Ratings for other psychometric properties were less than ture, climate, and related constructs; however, it also suggests a lack
“adequate.” Information was often available regarding cost, language of conceptual clarity with respect to the differentiation between
readability, and brevity. Information was less often available regard- molar organizational culture and molar organizational climate. Impli-
ing training burden and interpretation burden. Cost and language cations for measure development and refinement will be discussed.
readability generally exhibited “good” or “excellent” ratings, interpret-
ation burden generally exhibiting “minimal” ratings, and training bur- Reference
den and brevity exhibiting mixed ratings across measures. 1. Lewis CC, Proctor E, Brownson RC. Measurement issues in dissemination
Conclusions and implementation research. In: Brownson RC, Colditz GA, Proctor E,
Measures of readiness for implementation and its sub-constructs eds. Dissemination and implementation research in health: translating
used in mental health and behavioral healthcare are unevenly dis- science to practice. 2nd ed. New York: Oxford University Press; 2018.
tributed, exhibit unknown or low psychometric quality, and demon-
strate mixed pragmatic properties. This review identified a few
promising measures, but targeted efforts are needed to systematic- A70
ally develop and test measures that are useful for both research and A systematic review of outer setting measures in behavioral health
practice. Sheena McHugh1, Eric J. Bruns2, Jonathan Purtle3, Caitlin N. Dorsey4,
Kayne D. Mettert4, Cara C. Lewis4
1
Reference University College Cork, Cork, Ireland; 2University of Washington, Seattle,
1. Lewis CC, Proctor E, Brownson RC. Measurement issues in dissemination CA, USA; 3Drexel University, Philadelphia, PA, USA; 4Kaiser Permanente
and implementation research. In: Brownson RC, Colditz GA, Proctor E, Washington Health Research Institute, Seattle, CA, USA
eds. Dissemination and implementation research in health: translating Correspondence: Sheena McHugh ([email protected])
science to practice. 2nd ed. New York: Oxford University Press; 2018. Implementation Science 2020, 15(Suppl 2):A70
Background
A69 One of the main challenges to measurement of implementation-
Measuring organizational culture and climate: a systematic review relevant constructs is the mismatch between the level of measure-
Byron J. Powell1, Kayne D. Mettert2, Caitlin N. Dorsey2, Mark G. Ehrhart3, ment and the level of analysis [1-2]. This alignment is particularly
Gregory A. Aarons4, Bryan J. Weiner5, Cara C. Lewis2 challenging when measuring constructs relating to the outer setting,
1
Brown School, Washington University in St. Louis, St. Louis, MO, USA; given the predominance of self-report in implementation science.
2
Kaiser Permanente Washington Health Research Institute, Seattle, WA, The objective of this review is to assess the reliability, validity, and
USA; 3University of Central Florida, Orlando, FL, USA; 4University of practicality of measures of outer setting and its CFIR-delineated con-
California San Diego, San Diego, CA, USA; 5University of Washington, structs used in mental or behavioral healthcare.
Seattle, WA, USA Materials and Methods
Correspondence: Byron J. Powell ([email protected]) This review of outer setting measures follows the same study proto-
Implementation Science 2020, 15(Suppl 2):A69 col as the other systematic reviews initiated through this larger pro-
ject [3]. Phase I, data collection, occurred in five steps: a) search
Background string generation, b) title and abstract screening, c) full text review,
Organizational culture and climate have a long history in manage- d) construct assignment, and e) measure forward searches. Particular
ment research, and have been shown to impact implementation and to this review, during search string generation an additional level
clinical outcomes in mental health service settings. The purpose of was included for each of the CFIR constructs [4]; 1) Patient needs
this systematic review is to identify and evaluate the psychometric and resources, 2) Cosmopolitanism, 3) Peer Pressure, and 4) External
properties of measures of these constructs as they are defined by Policy & Incentives. Phase II, data extraction, consisted of coding rele-
the Consolidated Framework for Implementation Research. Specific- vant information to the nine psychometric rating criteria using the
ally, we aim to review measures of organizational culture, climate, Psychometric And Pragmatic Evidence Rating Scales (PAPERS) [3].
and its subconstructs including tension for change, compatibility, Phase III, data analysis, is underway. For each construct, frequencies
will be used to summarize the availability of psychometric informa- Results

tion for each PAPERS criterion for the measures for that construct. While most participants reported high intentions to use TNs, nearly
The median and the range of final ratings for each psychometric half reported that they did not use TNs in the last six months. The
PAPERS criterion will be used to summarize the psychometric initial survey identified beliefs related to TPB constructs, including
strength of the measures for that construct. clinician attitudes (e.g., TN may worsen symptoms), norms (e.g., care-
Results givers may disapprove), and self-efficacy (e.g., insufficient training).
Electronic searches yielded four measures of outer setting; four mea- CFIR-informed qualitative interviews yielded themes related to client
sures of patient needs and resources; eight measures of cosmopolit- and family characteristics (e.g., client reluctance, instability in care-
anism; one measure of peer pressure and six measures of external givers) and organizational factors (e.g., brief sessions).
policy and incentives. Five of these measures were unsuitable for rat- Conclusions
ing. Analysis of the psychometric properties of the remaining mea- Results indicate a discrepancy between intentions and TN use. Re-
sures if ongoing. sponses to the survey and qualitative interviews indicate that there are
Conclusions contextual factors that may moderate the relationship between inten-
The outer setting is the CFIR domain with the fewest measures iden- tions and behaviors. For example, organizational constraints, such as
tified. This paper advances implementation science and practice time limitations, may prevent TN use even among therapists with high
through consideration of how the outer setting should be conceptu- intentions. Results from this study highlight the importance of integrat-
alized and measured. Outer setting constructs may be more appro- ing theories that address multiple determinants of clinician behavior to
priately assessed at a system rather than individual level, and by identify potential targets for implementation strategies.
using direct measurement instead of latent variables.
References
References 1. Cohen JA, Mannarino AP. Trauma-Focused Cognitive Behavioural Therapy
1. Emmons KM, Weiner B, Fernandez ME, Tu SP. Systems antecedents for for children and parents. Child Adol Ment Health. 2008;13(4):158-162.
dissemination and implementation: a review and analysis of measures. 2. Allen B, Johnson JC. Utilization and implementation of Trauma-Focused
Health Educ Behav. 2012;39(1):87-105. Cognitive–Behavioral Therapy for the treatment of maltreated children.
2. Lewis CC, Proctor E, Brownson RC. Measurement issues in dissemination Child Maltreat. 2011;17(1):80-85.
and implementation research. In: Brownson RC, Colditz GA, Proctor E, 3. Ajzen I. The theory of planned behavior. Organ Behav Hum Dec Process.
eds. Dissemination and implementation research in health: Translating 1991;50(2):179-211.
science to practice. 2nd ed. New York: Oxford University Press; 2018. 4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC.
3. Lewis CC, Mettert KD, Dorsey CN, Martinez RG, Weiner BJ, Nolen E, Fostering implementation of health services research findings into
Stanick C, Halko H, Powell BJ. An updated protocol for a systematic practice: A consolidated framework for advancing implementation
review of implementation-related measures. Syst Rev. 2018;7(1):66. science. Implement Sci. 2009; 4: 50. doi:10.1186/1748-5908-4-50.
4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC.
practice: a consolidated framework for advancing implementation A72
science. Implement Sci. 2009;4(1):50. The relationship between therapist-driven adaptations to
evidence-based practices (EBP) and the extensiveness of EBP
strategy delivery in community implementation
A71 Stephanie H. Yu1, Lauren Brookman-Frazee2, Joanna J. Kim1, Miya L.
Applying the Theory of Planned Behavior and the Consolidated Barnett3, Anna S. Lau1
1
Framework for Implementation Research to understand therapists’ University of California, Los Angeles, Los Angeles, CA, USA; 2University
perceived barriers and facilitators to using trauma narratives of California, San Diego, San Diego, CA, USA; 3University of California,
Hannah Frank1, Briana Last2, Reem AlRabiah2, Jessica Fishman2, Brittany Santa Barbara, Santa Barbara, CA, USA
Rudd2, Hilary Kratz3, Colleen Harker2, Sara Fernandez-Marcote2, Kamilah Correspondence: Stephanie H. Yu ([email protected])
Jackson2, Rinad Beidas2 Implementation Science 2020, 15(Suppl 2):A72
1
Temple University, Philadelphia, PA, USA; 2University of Pennsylvania,
Philadelphia, PA, USA; 3La Salle University, Philadelphia, PA, USA Background
Correspondence: Hannah Frank ([email protected]) Community therapists adapt evidence-based practices (EBPs) to en-
Implementation Science 2020, 15(Suppl 2):A71 hance fit for their complex settings and clients when implemented
[1]. Yet, few studies have examined the potential implications
Background therapist-driven adaptations have for the quality of EBP delivery [2].
Trauma narratives (TN) are a critical component of trauma-focused We examined the extent to which different types of therapist-
cognitive-behavioral therapy (TF-CBT; [1]) yet therapists report using reported adaptations were associated with the extensiveness of EBP
TNs infrequently [2]. One causal theory that may explain the infre- strategy delivery in a system-driven implementation of multiple EBPs.
quent use of TNs is the Theory of Planned Behavior (TPB; [3]). The Materials and Methods
TPB states that intentions, which are informed by attitudes, subject- Data were drawn from an observational study investigating the sustain-
ive norms, and self-efficacy, are the strongest predictor of actual be- ment of six EBPs for youth in the Los Angeles County Department of Men-
havior. The TPB also acknowledges that certain contextual factors tal Health [3]. Community therapists (n=103) provided descriptions of any
need to be present in order for intentions to translate into actual be- adaptations they made in 680 sessions with 273 clients. Trained coders
havior. Implementation science frameworks, such as the Consoli- rated the extensiveness of EBP strategy delivery via the EBP Concordant
dated Framework for Implementation Research (CFIR; [4]), provide Care Assessment (ECCA) from session recordings [3]. We examined how
insight into the types of contextual factors that may affect clinician different types of therapist adaptations were associated with ECCA exten-
behavior (e.g., organizational variables). This study aimed to identify siveness ratings. Adaptations were categorized as: 1) modifying presenta-
barriers to TN use by integrating a causal theory of behavior change tion, 2) integrating (integrating components, combining practices,
(TPB) and a widely used contextual framework (CFIR). providing psychoeducation), 3) extending (repeating components, length-
Materials and Methods ening), 4) reducing (removing, reordering, or shortening components),
Sixty-five mental-health therapists working in community settings and 5) other (adaptations that could not be coded within our framework
and trained through a city-wide TF-CBT initiative participated by due to lack of fit) [4]. Furthermore, we examined these relationships
completing a survey about their use of and beliefs about TNs. Con- through an augmenting (modifying presentation, integrating, extending)
tent analysis was conducted to identify common beliefs about TNs. A vs. reducing (removing, reorder, shortening) vs. “other” framework [4].
subset of participants (n=17) completed in-depth qualitative inter- Results
views focused on perceptions of TNs. Qualitative interviews were an- Preliminary analysis revealed that the overall number of therapist ad-
alyzed using an integrated approach informed by the CFIR. aptations did not significantly predict extensiveness ratings (β = .067,
p = .059). However, specific adaptations were related to extensive- intentions varied widely when clinicians reported on specific CBT
ness. Modifying presentation was associated with higher extensive- components. In general, higher intentions were observed for CBT
ness (β = .098, p = .046), while “other” adaptations were associated components that were simpler and part of the session structure (e.g.,
with lower extensiveness (β = -.119, p = .034). When compared reviewing homework vs. exposure therapy). Additional analyses will
through an augmenting vs. reducing vs. “other” framework, “other” examine clinician characteristics as predictors of intentions.
adaptations predicted lower extensiveness (β = -.127, p = .023). Conclusions
Conclusions Findings highlight variability in clinician intentions across specific ele-
Quality of EBP delivery may be robust to some types of therapist- ments of CBT and suggest that CBT implementation strategies may
driven adaptations. In fact, adaptations that tailor the presentation of need to be tailored as a function of specific intervention compo-
interventions were associated with more extensive EBP delivery. In nents. Results also suggest that implementation efforts may benefit
contrast, “Other” adaptations were more likely to be associated with from dismantling complicated interventions, highly specifying the
lower EBP extensiveness and thus may represent “drift.” Additional practitioner behavior of interest, and tailoring implementation strat-
analyses to further characterize the content of the “other” adapta- egies to each one.
tions will be included in the proposed presentation.
References
References 1. Fishman J, Beidas RS, Reisinger E, Mandell DS. The utility of measuring
1. Stirman SW, Gamarra JM, Bartlett BA, Calloway A, Gutner CA. Empirical intentions to use best practices: a longitudinal study among teachers
examinations of modifications and adaptations to evidence-based psy- supporting students with autism. J School Health. 2018;88(5):388-95.
chotherapies: methodologies, impact, and future directions. Clin Psychol. 2. Novins DK, Green AE, Legha RK, Aarons GA. Dissemination and
2017;24:396-420. implementation of evidence-based practices for child and adolescent mental
2. Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, health: a systematic review. J Am Acad Child Adol Psych. 2013;52(10):1009-25.
Hartman M, Fernandez ME, Mullen PD. A systematic review of adapta- 3. McHugh RK, Barlow DH. The dissemination and implementation of
tions of evidence-based public health interventions globally. Implement evidence-based psychological treatments: a review of current efforts. Am
Sci. 2018;13:125. Psychol. 2010;65(2):73.
3. Lau AS. Brookman-Frazee L. The 4KEEPS study: identifying predictors of 4. Smith MM, McLeod BD, Southam-Gerow MA, Jensen-Doss A, Kendall PC,
sustainment of multiple practices fiscally mandated in children’s mental Weisz JR. Does the delivery of CBT for youth anxiety differ across research
health services. Implement Sci. 2016;11:31. and practice settings?. Behav Ther. 2017;48(4):501-16.
4. Lau A, Barnett M, Stadnick N, Saifan D, Regan J, Stirman SW, Roesch S. 5. Finley EP, Noël PH, Lee S, et al. Psychotherapy practices for veterans with
Brookman-Frazee L. Therapist report of adaptations to delivery of evidence- PTSD among community-based providers in Texas. Psychol Serv.
based practices within a system-driven reform of publicly funded children’s 2018;15(4):442.
mental health services. J Consult Clin Psychol. 2017;85(7):664-675.
A74
A73 A comparison of consultant effects, activities, and perceptions on
One size does not fit all: clinician intentions to implement therapist fidelity and patient treatment outcomes
cognitive-behavioral therapy vary by specific component Heidi La Bash1, Norman Shields2, Tasoula Masina3, Kera
Emily Becker-Haimes, Jessica Fishman, Torrey Creed, Courtney Benjamin Swanson1, Jiyoung Song1, Clara Johnson1, Matthew Beristianos1, Erin
Wolk, Nicholas Affrunti, Danielle Centeno, David Mandell Finley4,5, Vanessa Ramirez5, Jeanine Lane3, Michael Suvak6, Candice
University of Pennsylvania, Philadelphia, PA, USA Monson3, Shannon Wiltsey Stirman1
1
Correspondence: Emily Becker-Haimes ([email protected]) National Center for PTSD, VA Palo Alto Health Care System, Palo Alto,
Implementation Science 2020, 15(Suppl 2):A73 CA, USA; 2Veterans Affairs Canada, West Montreal, Quebec, Canada;
3
Ryerson University, Toronto, Ontario, Canada; 4South Texas Veterans
Background Health Care System, San Antonio, TX, USA; 5University of Texas Health
Developing implementation strategies to increase clinicians’ use of Science Center at San Antonio, San Antonio, TX, USA; 6Suffolk University,
evidence-based practices (EBPs) is important for improving the qual- Boston, MA, USA
ity of mental health care. Our prior work with school-based services Correspondence: Heidi La Bash ([email protected])
for youth with autism demonstrated that studying practitioners’ in- Implementation Science 2020, 15(Suppl 2):A74
tentions to use specific EBP components rather than their intentions
to use broader intervention protocols, may point to levers by which Background
to tailor implementation strategies as a function of the specific EBPs Research has demonstrated that workshops alone do not lead to suf-
themselves (e.g., their salience, complexity) [1]. We extend this work ficient skill in delivering evidence-based psychotherapies (EBP), and
by examining variability in community clinicians’ intentions to use that strategies such as follow-up consultation are needed. Yet, there
different components of cognitive-behavioral therapy (CBT). CBT has is little research to inform how to best provide consultation to ensure
long been a target of implementation efforts [2-3] yet remains under- sustained, high-quality delivery of EBPs.
utilized in community settings [4-5]. CBT comprises multiple, distinct Materials and Methods
components that vary in their complexity, ranging from relatively The parent randomized controlled implementation trial assessed the
simple (e.g., homework review) to more complex interventions (e.g., impact of three post-cognitive processing therapy (CPT) training
cognitive restructuring). Examining how intentions vary across spe- workshop conditions (No consultation, Standard consultation without
cific CBT components may yield insights into how to tailor imple- session audio review, Consultation with audio review) on the patient
mentation strategies to increase CBT use in community settings. (N=188) PTSD treatment outcomes of participating therapists (n=134)
Materials and Methods in over 30 routine care settings in Canada. The current mixed-
Community mental health providers (N = 149, mean age = 40.8 years, methods study examines associations between consultation activities
57.7% female) who had received intensive training and consultation in and therapist fidelity and patient treatment outcomes, as well as the
CBT were surveyed about their intentions to use six key elements of accuracy of consultant perceptions of the skill and engagement of
CBT interventions (exposure therapy, cognitive restructuring, behavioral therapist consultees. Consultation occurred weekly for six months.
activation, planning homework, reviewing homework, and agenda- After each recorded call, consultants (n=5) completed a post-call
setting) for seven different clinical presentations. Clinicians also re- checklist of strategies and rating of perceived levels of enthusiasm,
ported on their intentions to use “EBP” broadly. Demographic and clin- skill, and participation for each consultee. A subset of therapists (n=
ical background characteristics also were collected. 30) were interviewed at the end of the consultation phase.
Results Results
Analyses are ongoing. Nearly all clinicians reported high intentions to While there was variability, three primary categories of activities
use “EBPs” broadly across all seven clinical presentations. However, emerged: case conceptualization and intervention planning, feedback
on fidelity, and distractions/technical difficulties. Similarities and differ- References

ences in consultant and therapist perceptions of consultation activities 1. Zimmerman L, Lounsbury D, Rosen C, Kimerling R, Trafton J. Lindley S.
will be presented. Additionally, analyses revealed evidence of a consult- Participatory system dynamics modeling: Increasing stakeholder
ant effect on therapist treatment adherence (B=0.439, SE=0.171, engagement and precision to improvement implementation. Adm Policy
p=.012), but not competence (-B=0.341, SE=0.247, p=.168). This will be Ment Health. 2016;43(6):834-849. doi:10.1007/s10488-016-0754-1
explored in relation to consultation condition and strategies, including 2. Zimmerman L, Lounsbury D, Rosen C, Kimerling R, Holbrook A, Hong S,
whether different consultants engaged in specific activities more fre- Branscomb J, Kibbe D, Park S, Kramer R, Barlow DC, Mushiana S, Azevedo K,
quently. Finally, we found that while consultants perceived overall im- Yang J, Trafton J, Lindley S, Rust T. 2018, December. Participatory system
provements over the course of consultation (b = 0.03, t = 6.12, p < .01), dynamics for high quality VA addiction and mental health care. Oral
their ratings of therapist skill did not predict clinician adherence (b = presentation in L. Zimmerman (Chair), Participatory modeling approaches
0.03, t = 0.46, p = .65), competence (b = 0.03, t = 0.31, p =.76), or pa- to implementation science, 11th Annual Conference on the Science of
tient PTSD symptom change (b = -.12, t = -0.31, p = .76). It is possible Dissemination and Implementation in Health, Washington, D.C.
that cognitive biases (e.g., halo effect) may reduce the accuracy of con- 3. Sterman JD. Learning from evidence in a complex world. Am J Public
sultant perceptions. Health. 2006;96:505–514.
Conclusions 4. Hovmand PS. Community Based System Dynamics. New York, NY:
Practical implications of this and the other study findings will be pre- Springer New York; 2014.
sented in a broader discussion of barriers/facilitators of EBP 5. Hovmand PS, Rouwette EAJA, Andersen DF, Richardson GP, Kraus A.
sustainment. Scriptapedia 4.0.6. 2013.
Trial Registration: Clinicaltrials.gov NCT02449421
A75 A76
From blank page to local optimization: participatory systems Building implementation capacity through development of a
modeling to improve local evidence based practice implementation coaching network
David Lounsbury1, Debra Kibbe2, James Rollins3, Lindsey Zimmerman4 Kristen Miner, Emily Bilek, Jennifer Vichich, Shawna Smith, Elizabeth
1
Albert Einstein College of Medicine, Yeshiva University, Bronx, NY, USA; Koschmann
2
Georgia Health Policy Center, Georgia State University, Atlanta, GA, USA; University of Michigan, Ann Arbor, MI, USA
3
Takouba Security LLC, Seattle, WA, USA; 4National Center for PTSD, VA Correspondence: Kristen Miner ([email protected])
Palo Alto Health Care System, Palo Alto, CA, USA Implementation Science 2020, 15(Suppl 2):A76
Correspondence: David Lounsbury ([email protected])
Mood and anxiety disorders affect approximately 30% of youth [1].
Background Cognitive Behavioral Therapy (CBT) is an evidence-based treatment
This panel is for implementation practitioners interested in systems sci- for these disorders, but only a fraction of adolescents in need have
ence and participatory modeling approaches to evidence-based practice access to high quality treatment. Access could be substantially in-
(EBP) implementation. Panelists will review participatory development of creased if school professionals (SPs) were trained to deliver CBT.
“Modeling to Learn (MTL),” a program for improving EBP implementation However, typical professional development and clinical training op-
in the Veterans Health Administration (VA) [1-2]. Study of MTL is under- portunities are often unsuccessful because they lack post-training
way to determine its effectiveness increasing delivery of evidence-based support necessary for producing sustained behavioral change. TRAILS
addiction and mental health care (R01DA046651). Preliminary statistical is an implementation and training model designed to increase
process control analyses indicate pilot clinics demonstrated a three stand- utilization of CBT among SPs. In addition to clinical training and re-
ard deviation increase in EBP reach and maintained improvement for 12 sources, TRAILS is unique in that it also provides in-person coaching
and 8 months, respectively (R21DA042198). to support SPs as they co-lead student CBT skills groups.
Using the MTL example, this panel answers commonly asked ques- To build capacity for large-scale and sustainable implementation of
tions about systems science approaches to implementation through CBT, TRAILS established a statewide network of expert “coaches”,
demonstration. Four years ago, panelists began learning with front- recruiting primarily from Community Mental Health (CMH) agencies
line multidisciplinary teams about determinants of local reach of across Michigan. Participation was incentivized via provision of train-
evidence-based psychotherapies and pharmacotherapies (EBPs). Sup- ing and CEUs at no cost to potential coaches and up to 10 (non-
port for modeling to identify locally tailored implementation plans coaching) clinicians per partner agency. Coaches participated in a
grew among VA stakeholders, and each panelist joined the project to daylong CBT training, followed by 12 weeks of individual case con-
contribute systems modeling expertise: Dr. Lounsbury as an NIH- sultation from an expert clinician. Coaches were assessed pre- and
funded researcher, Ms. Kibbe as a public health facilitator, and Col. post- consultation and successful completers were recommended for
Rollins (Ret.) as an online simulation user interface developer. subsequent training in the TRAILS coaching protocol.
Results Results
Panelists will present participatory modeling activities, linking them to TRAILS trained five cohorts of coaches over the course of two
free, online open science resources. Dr. Lounsbury will describe initial years. Of 125 trainees completing consultation, 108 were recom-
qualitative group model building exercises, developed in the field of sys- mended for coaching based on clinical skill, and 84 were ultim-
tem dynamics [4-5]. These activities illustrate how to determine the right ately trained in the TRAILS coaching protocol. TRAILS Coach
modeling problem via participatory engagement. Ms. Kibbe will describe training led to improvement in CBT expertise (p<0.001) and fre-
the participatory principles and pragmatic constraints used to refine MTL quency of use (p<0.001), with 87% of coaches reporting feeling
online facilitation resources for teams to develop systems thinking skills “extremely satisfied” with their training. TRAILS Coaches now span
at national scale [3]; this includes the MTL session guides and fidelity 63 of Michigan’s 83 counties and have been paired with 47
checklist. Col. Rollins will describe design iterations to produce an inter- schools across Michigan as part of a NIMH clinical trial.
face for frontline teams to simulate improvement scenarios using team Conclusions
data. Attendees will be provided access to a demonstration website de- Large-scale implementation efforts often require development of
veloped to help users understand how simulation makes local EBP bar- new infrastructure to ensure capacity to support implementation
riers and facilitators more transparent and locally manageable. strategy deployment and long-term sustainability. By creating a
Conclusions statewide network of coaches, TRAILS has built the infrastructure ne-
As discussant, Dr. Zimmerman will synthesize the activities described by cessary to support SPs in learning and delivering CBT to students in
panelists in relation to the learning objectives, to help implementation need and provides one model for feasible creation of statewide im-
practitioners and intermediaries get started with participatory modeling. plementation infrastructure.
Reference A78
1. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, What works best in practice? the effectiveness of ‘real-world’
Benjet C, Georgiades K, Swendsen J. Lifetime prevalence of mental facilitation strategies in overcoming evidence-based barriers to
disorders in U.S. adolescents: results from the National Comorbidity implementation
Survey Replication—Adolescent Supplement (NCS-A). J Am Acad Lydia Moussa, Katarzyna Musial, Simon Kocbek, Victoria Garcia Cardenas
Child Adolesc Psychiatry. 2010;49(10):980-9. doi:10.1016/j.jaac.2010. University of Technology Sydney, Sydney, NSW, Australia
05.017. Correspondence: Lydia Moussa ([email protected])
A77 Background
The parent engagement in evidence-based services questionnaire: Research has shed light into factors affecting the implementation of in-
advancing our understanding of parental intentions for engaging novations in practice as well as facilitation strategies that can be utilised
in evidence-based practice during the implementation of these innovations. Change facilitators,
Spencer Choy, Jaime Pua Chang, Brad Nakamura however, often use a ‘trial and error’ approach when determining the
University of Hawaii at Manoa, Honolulu, HI, USA best strategy to overcome the particular barrier identified. This leads to
Correspondence: Spencer Choy ([email protected]) loss of time, resources and a team’s reduced motivation to successfully
Implementation Science 2020, 15(Suppl 2):A77 implement the innovation. To overcome this challenge, we developed
a machine-learning tool that connects evidence-based barriers with the
Background most effective, pragmatic change facilitation strategies, as trialed by
The Parent Engagement in Evidence-Based Services (PEEBS) [1] change facilitators in the real world.
is a newly developed questionnaire that aims to assess parent Materials and Methods
consumer perspectives about evidence-based practices through A 2-year change program was facilitated across 19 pharmacies
the theory of planned behavior, a widely-applied model that around Australia by six facilitators. Facilitators identified barriers
suggests parental attitudes, perceived behavioral control (PBC), to implementation and recorded the strategies they used to
and subjective norms (SN) shape behavioral intentions. Towards overcome these barriers. The barriers were coded according to
the larger goal of better understanding factors associated with implementation factors from the Consolidated Framework of Im-
parental behavioral intentions for utilizing such services, we plementation Research [1] and the Theoretical Domains Frame-
present data with a community parent sample of the PEEBS’ work. Strategies were coded according Dogherty et al. facilitation
factor structure, internal consistencies, and relations with related strategy taxonomy [2]. To determine the most effective facilita-
instruments. tion strategies, a decision forest [3] algorithm was developed.
351 parents (75.8% female; M = 40.4 years old, SD = 7.6; 62.3% We collected 1131 data points from six facilitators, which were cate-
Asian) recruited from 15 community outreach efforts in Hawaii gorised into 36 barriers to implementation and 111 unique change
completed the PEEBS, the Family Empowerment Scale (FES) [2], facilitation strategies. The Decision Forest algorithm highlighted the
and the Parental Attitudes Toward Psychological Services Inven- effectiveness of the facilitation strategies according to the strategy
tory (PATPSI) [3]. Exploratory factor analysis was conducted with ‘resolve rate’ (RR). The most frequent barrier in the pharmacy practice
principal axis factoring and oblique rotation, Cronbach alpha setting was the ‘inability to plan for change’ (n=184). The strategies
coefficients were used to calculate internal consistencies, and used to overcome this barrier were to: a) ‘Manage the different the
Pearson correlations were computed to investigate convergent requirements of each discipline/ role and create ownership’ (RR=
validity. 84.23%), b) ‘Provide training’ (RR= 83.30%) c) ‘Adapt area of focus to
Results change’ (RR= 81.17) d) ‘Assist the group to develop ideas and solve
Exploratory factor analysis suggested a five-factor structure (50 problems’ (RR= 80.64%).
items, alpha = .86). Items grouped along the dimensions of SN Conclusions
(11 items, alpha = .80), treatment barriers (10 items, = .71), treat- By understanding that the most prevalent barrier to implementa-
ment knowledge (6 items, alpha = .72), evidence-informed action tion was an ‘inability to plan for change’, this provides pharmacy
(11 items, alpha = .83), and PBC (12 items, alpha = .83), that practice policy makers, tertiary educators, researchers and change
accounted for 38.07% of the total variance. Regarding Pearson bi- facilitators an idea of where to target strategies to not only
variate correlations, PEEBS’ PBC was significantly and positively overcome, but prevent this barrier from occurring. This tool can
associated with evidence-informed action (r = .50, p < .01) and be reproduced to understand implementation barriers specific to
SN (r = .35, p < .01). As expected, the FES family scale was sig- other industries and the most effective strategies to overcome
nificantly and positively associated with the PEEBS’ knowledge these.
factor (r = .42, p < .01), PBC (r = .32, p < .01), and evidence-
informed action (r = .17, p < .01). PATPSI’s help-seeking inten- References
tions were significantly and positively correlated with the PEEBS’ 1. Damschroder L, Hagedorn H. A guiding framework and approach for
PBC (r = .37, p < .01), evidence-informed action (r = .33, p < .01), implementation research in substance use disorders treatment. Psychol
knowledge (r = .29, p < .01), and subjective norms (r = .22, p < Addict Behav. 2011;25(2):194-205. doi:10.1037/a0022284
.01). 2. Dogherty E, Harrison M, Graham I. Facilitation as a role and process in
Conclusions achieving evidence-based practice in nursing: a focused review of con-
Based on promising psychometric results, the PEEBS appears to be a cept and meaning. Worldviews Evid Based Nurs. 2010;7(2):76-89.
potentially useful instrument to understand parent consumers. doi:10.1111/j.1741-6787.2010.00186.x
3. Khalilia M, Chakraborty S, Popescu M. Predicting disease risks from highly
imbalanced data using random forest. BMC Med Inform Decis Mak.
References
2011;11(1). doi:10.1186/1472-6947-11-51
1. Chang JP, Orimoto TE, Selbo-Bruns A, Choy SKJ, & Nakamura BJ. Applica-
tion of the Theory of Planned Behavior to caregiver consumer engage-
ment in evidence-based services. (under review). A79
2. Koren PE, DeChillo N, Friesen BJ. Measuring empowerment in families Implementation strategies of a co-designed physical activity
whose children have emotional disabilities: a brief questionnaire. Rehab program for older adults
Psychol. 1992;37(4):305-321. Erica Lau, Joanie Sims-Gould, Samantha Gray, Heather McKay
3. Erlanger EA. The parental attitudes toward psychological services University of British Columbia, Vancouver, British Columbia, Canada
inventory: Adaptation and development of an attitude scale. Comm Correspondence: Erica Lau ([email protected])
Ment Health J. 2012;48(4):436-449. Implementation Science 2020, 15(Suppl 2):A79
Background Pay for Success (PfS), also known as social impact bonds, is one such
Despite the known health benefits of physical activity (PA), 87% of strategy [1]. Under PfS, governments may leverage private or philan-
Canadian older adults do not meet recommended PA guidelines. thropic upfront capital to fund interventions, and subsequently repay
Community-based PA interventions show promise, but few were those funders based on outcomes. However, there are notable con-
scaled-up. With partners, the Active Aging Research Team (AART) co- siderations as it relates to implementation of this strategy, as de-
created Choose to Move (CTM), a 6-month, choice-based health pro- scribed by Lowe et al. (2019) [2]. According to their model, it is easier
motion intervention that aims to improve older adults’ social con- to achieve agreement at the macro-policy levels, but challenges arise
nectedness and mobility through PA. During small scale-up, two in meso- and micro- levels.
community delivery partner organizations delivered 56 CTM pro- In 2015, the Colorado state legislature approved a law that enabled the
grams in 26 urban communities across BC. We previously demon- state to enter into PfS arrangements. Shortly thereafter, a “Call for
strated that CTM effectively improved PA, mobility and social Innovation” was released from the Governor’s Office to identify poten-
connectedness in 458 older adults [1]. The objective of this study tial PfS projects to improve outcomes for at-risk teens. The present pro-
was to evaluate effectiveness of CTM implementation strategies to ject was one of three selected, and focuses on bringing an evidence-
guide broad scale-up (175 programs) of CTM. based intervention, Multisystemic Therapy [3], to under-served regions
Materials and Methods of the state. This presentation includes the perspectives of a state part-
Grounded in implementation frameworks (e.g., QIF [2], Powell et. al. ner and the University-based entity implementing the project.
[3]), CTM adopted eight key implementation strategies: 1. develop Materials and Methods
strong community partnerships; 2. develop an implementation blue- The Lowe et al. (2019) [2] framework is used to contextualize the PfS
print; 3. ongoing implementation monitoring; 4. promote adaptabil- structuring of the initiative, which required alignment of funders,
ity; 5. provision of program materials and tools; 6. centralized state partners, evaluators, and the project-based implementation
training and technical assistance; 7. convene advisory committees, team. This is the first PfS initiative that partnered with a non-profit,
and 8. a staged implementation scale-up approach. To assess effect- private university as the primary fiscal agent.
iveness of implementation strategies, we measured four implementa- Results
tion outcomes (reach, participant responsiveness, quality, and This presentation describes the successful launch of the Colorado-
delivery partners’ perceptions of the strategy). We administered sur- based PfS initiative in December, 2018. Three cohorts, each contain-
veys and conducted interviews with CTM participants (n=42) and de- ing two MST teams, are being rolled out over the next two years.
livery partners across four levels of influence (19 decision makers, 6 The evaluation, on which the success payments are connected, uses
recreation managers, 27 recreation coordinators, 23 activity coaches) a propensity score matching procedure because of the relatively
at mid- and post-intervention. lower population density in the service areas. Over the course of the
Results project, we anticipate over 600 families to be served.
CTM implementation strategies were effective across all levels of Conclusions
evaluation. Of 458 participants, 82% attended ≥75% of group meet- Although complicated to enact, PfS may become an important strat-
ings; 95% completed ≥70% of check-ins (reach); 75.3% were satisfied egy to support large scale dissemination. Strategies to simplify the
with CTM (participant responsiveness). From interviews, CTM was im- contracting process and support the meso- and micro-policy tensions
plemented with quality. Participants’ had very positive perceptions of is necessary for broad-scale uptake.
their ACs. Delivery partners noted that intervention materials were
appropriate and CTM was flexible and easy to implement. Central References
support (e.g., training, integration) provided by AART was instrumen- 1. Warner ME. Private finance for public goods: Social impact bonds. J Econ
tal to effectively implementing CTM. Policy Reform. 2013;16(4):303-319.
Conclusions 2. Lowe T. Kimmitt J, Wilson R, Martin M, Gibbon J. The institutional work of
Implementation strategies must be adapted to provide “best fit” for creating and implementing Social impact bonds. Policy Politics.
the delivery context. Our findings support a suite of implementation 2019;47(2);353-370.
strategies that promoted scale up a health promotion intervention 3. Henggeler SW, Pickrel SG, Brondino MJ. Multisystemic treatment of
that other interventions can adopt, modify and evaluate in future. substance-abusing and-dependent delinquents: Outcomes, treatment fi-
delity, and transportability. Ment Health Ser Res. 1999;1(3):171-184.
References
1. McKay H, Nettlefold L, Bauman A, Hoy C, Gray SM, Lau E, Sims-Gould J.
Implementation of a co-designed physical activity program for older A81
adults: Positive impact when delivered at scale. BMC Public Health. Using common elements and co-creation to enhance
2018;18(1):1289. implementability of evidence-informed interventions: example
2. Meyers DC, Durlak JA, Wandersman A. The quality implementation from an academic intervention in Norwegian child welfare
framework: A synthesis of critical steps in the implementation process. Thomas Engell1, Benedicte Kirkøen1, Karianne Thune Hammerstrøm1, Hege
Am J Comm Psychol. 2012;50(3-4):462-480. Kornør2, Kristine Horseng Ludvigsen1, Kristine Amlund Hagen3
1
3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu Regional Centre for Child and Adolescent Mental Health, Trondheim,
MM, Proctor EK, Kirchner JE. A refined compilation of implementation Norway; 2Norwegian Institute of Public Health, Oslo, Norway;
3
strategies: Results from the Expert Recommendations for Implementing Norwegian Center for Child Behavioral Development, Oslo, Norway
Change (ERIC) project. Implement Sci. 2015;10:21. Correspondence: Thomas Engell ([email protected])
A80 Background
Innovative funding to achieve reach: pay for success Implementation and sustainment of effective interventions remains a
Suzanne E. U. Kerns1, Mollie Bradlee2 struggle across public services, especially in settings with limited
1
University of Denver, Denver, CO, USA; 2Colorado Office of Children, readiness for implementation. Increasing the implementability of in-
Youth, and Families, Denver, CO, USA terventions can facilitate more successful implementation. There is
Correspondence: Suzanne E. U. Kerns ([email protected]) growing interest in identifying and studying discrete elements that
Implementation Science 2020, 15(Suppl 2):A80 are common across interventions for the purpose of hypothesis gen-
eration, intervention optimization and re-design, and implementation
Background in practice. We combine common elements methodology with col-
Large scale dissemination and implementation of evidence-based in- laborative design approaches to develop and test implementable
terventions is limited by funding. Further, agencies are often required evidence-informed interventions tailored to individual and contextual
to shoulder the burden of costs while monetary benefits are realized needs. This session will present preliminary results from a common
in other service sectors. Thus, novel funding strategies are warranted. elements-based academic intervention for children in child welfare.
Materials and Methods strategies delivered (indexed by both observer or therapist report). For
We used common elements methodology to identify common prac- parent challenges, the relationship was nonsignificant using therapist-
tice-, process-, and implementation elements in systematically reviewed reported EBP delivery. In contrast, parent engagement challenges were
academic interventions for children at risk. We compared frequencies positively associated with observer-rated therapist delivery of EBP strat-
of the most common elements and combinations in effective interven- egies (b=.15; p=.04). This relationship seems to be driven by parents ex-
tions with frequencies in ineffective interventions. Using facilitated co- pressing concerns about the relevance/acceptability/helpfulness of the
creation with stakeholders, practitioners and user-representatives, we EBP.
aimed to unify perspectives of researchers, implementers, managers, Conclusions
practitioners, and end-users to develop an implementable academic Overall, these findings imply that youth/parent behaviors that we
intervention based on the identified common elements. We developed may conceptualize as engagement challenges do not necessarily de-
dynamic fidelity monitoring encouraging flexible use of elements and rail therapists from EBP delivery in session. Future analyses will fur-
adaptations, and we co-created blueprints for implementing, evaluat- ther examine the relationship thematically by types of engagement
ing, and sustaining the intervention. We are now conducting a mixed- challenges. Discussion will focus on therapist perceptions of client
methods hybrid pragmatic trial where we assess the child welfare con- behaviors as engagement challenges, and whether they can be
texts’ readiness for implementation, intervention implementability deemed opportunities for alliance building.
(feasibility, appropriateness, acceptability, usability), implementation
quality, and intervention effectiveness. References
Results 1. Chu BC, Kendall PC. Positive association of child involvement and
We included 30 effective and 6 ineffective academic interventions for treatment outcome within a manual-based cognitive-behavioral treat-
children at risk in a systematic review. We identified 62 practice element for children with anxiety. J Consult Clin Psychol. 2004;72(5):821-829.
ments, 49 process elements, and 34 implementation elements used doi:10.1037/0022-006X.72.5.821.
in the interventions. Frequency count values (FVs; inclusion in effect- 2. Becker KD, Boustani M, Gellatly R, Chorpita BF. Forty years of
ive vs ineffective interventions) were calculated for each element engagement research in children’s mental health services:
and commonly used combinations of elements. Elements and combi- multidimensional measurement and practice elements. J Clin Child
nations with the highest FVs were used in facilitated co-creation to Adolesc Psychol. 2018;47(1):1-23. doi:10.1080/15374416.2017.1326121.
develop the intervention Enhanced Academic Support (EAS). Prelim-
inary mixed methods results on the implementability of EAS and as-
sociations with readiness will be presented at the conference. A84
Conclusions Therapist characteristics as predictors of perceptions of Evidence-
Combining common elements methodology with collaborative ap- Based Assessment (EBA)
proaches for intervention design can be a viable approach for devel- Kenny Le1, Lauren Brookman-Frazee2, Joyce Lui1, Mary Kuckertz2, Anna
oping implementable interventions tailored to individual and Lau1
1
contextual needs University of California, Los Angeles, Los Angeles, CA, USA; 2University
of California, San Diego, La Jolla, CA, USA
Correspondence: Kenny Le ([email protected])
A82 Implementation Science 2020, 15(Suppl 2):A84
Examining the relationship between client engagement challenges
and community therapists’ delivery of evidence-based strategies Background
to youth and their caregivers Evidence-based assessment (EBA) can improve clinical outcomes, and it
Blanche Wright1, Anna Lau1, Joanna Kim1, Resham Gellatly1, Mary is embedded in many evidence-based practices (EBP) [1]. However, EBA
Kuckertz2, Lauren Brookman-Frazee2 is not well implemented in community practice due to perceived low
1
University of California, Los Angeles, Los Angeles, CA, USA; 2University usefulness [2]. Additionally, therapist characteristics such as non-
of California, San Diego, La Jolla, CA, USA psychology disciplines and years of practice may be associated with
Correspondence: Blanche Wright ([email protected]) negative EBA attitudes [3]. Research is needed on factors that predict
Implementation Science 2020, 15(Suppl 2):A82 EBA use in community practice and how attitudes change with expos-
ure to EBP. The current study examined how therapist characteristics,
Background use of EBA within the delivery of EBPs, and interactions between these
In the delivery of youth-focused evidence-based practices (EBPs), variables may be associated with subsequent perceptions of EBA.
high levels of client engagement have been positively associated Materials and Methods
with improved clinical outcomes [1]. In the literature, engagement Therapists (n=117) in an initial survey reported on their use of clinical
has been primarily indexed by attendance [2], and more refined dashboards, an EBA strategy for repeated assessments of client out-
qualitative measures of engagement are needed. Thus, in the current comes to inform treatment planning. They reported on their percep-
study, we used an observational coding system to examine specific tions of EBA in another survey approximately 10 months later.
in-session client engagement challenges within a county-wide imple- Separate linear regression models were conducted for each of the
mentation of multiple EBPs in youth mental health services. four dimensions of EBA perceptions: clinical utility, practicality, bene-
Materials and Methods fit to clients and treatment, and harm to clients.
The aims of the study were twofold: (1) characterize the frequency of client Results
engagement challenges and (2) examine whether these challenges are asso- Results showed that dashboard use significantly predicted percep-
ciated with the extent to which community therapists deliver EBP strategies. tions of clinical utility (b=-0.100, p=0.006). Theoretical orientation
Extensiveness of EBP strategies was measured via therapist-report and moderated the relationship between dashboard use and clinical util-
observer-ratings. The sample included 103 therapists who provided record- ity (b=0.107, p=0.028). For therapists who did not identify as having
ings of 680 sessions in which they delivered an EBP to 273 youths (Mean a cognitive/behavioral orientation in their practice, more extensive
age=9.75 years). Using observational coding, in-session youth and parent en- dashboard use predicted lower perceptions of clinical utility of EBA.
gagement challenges (i.e., client expressed concerns, refusal to participate in For other EBA perceptions, years of practice significantly predicted
activities) were measured with good reliability (Mean ICC=.64). perceived practicality (b=0.037, p=0.020), where longer years of prac-
Results tice predicted positive perceptions of EBA. Therapist discipline also
Across both youth only and youth + parent sessions, 66.29% of sessions significantly predicted perceived harm of EBA for clients (b=-0.495,
had ≥ 1 engagement challenge occur. In parent only and youth + par- p=0.022), where having a psychology discipline predicted lower per-
ent sessions, engagement challenges were observed in fewer sessions ceived harm relative to other disciplines.
(30.47%). When examining the association between engagement chal- Conclusions
lenges and therapist delivery of EBP strategies, there was no significant These results replicate therapist characteristics as important predic-
association with occurrence of youth engagement challenges and EBP tors of EBA perceptions and suggest that on-the-job use of EBA
strategies can predict future perceptions. However, for some clini- 2. Starin AC, Atkins MS, Wehrmann KC, Mehta T, Hesson-McInnis MS,
cians (non-cognitive/behavioral), mandated use of EBA may further Marinez-Lora A, Mehlinger R. Moving science into state child and adoles-
entrench negative perceptions. Non-psychologists and less experi- cent mental health systems: Illinois’ evidence-informed practice initiative.
enced therapists perceived EBA as burdensome and more harmful to J Clin Child Adolesc Psychol. 2014;43(2):169–178.
therapeutic process perhaps due to inefficient training or support. 3. Aarons GA, Farahnak LR, Ehrhart MG. Leadership and strategic
Future studies could examine implementation strategies that may organizational climate to support evidence-based practice implementation.
promote positive perceptions of EBA and positive outcomes of EBA In: Beidas RS, Kendall PC, eds. Dissemination and implementation of
use for clients. evidence-based practices in child and adolescent mental health. New York,
NY: Guilford Press; 2014: p. 82–97.
References 4. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of
1. Scott K, Lewis CC. Using Measurement-Based Care to enhance any treat- evidence-based practice implementation in public service sectors. Adm
ment. Cogn Behav Pract. 2015; 22(1):49-59. doi:10.1016/J.CBPRA.2014.01.010 Policy Ment Health. 2011;38(1):4–23.
2. Jensen-Doss A. Practice involves more than treatment: How can 5. Anakwenze U, Zuberi D. Mental health and poverty in the inner city.
evidence-based assessment catch up to evidence-based treatment? Clin Health Soc Work. 2013;38(3):147-157.
Psychol Sci Pract. 2011;18(2):173-177. doi:10.1111/j.1468-2850.2011.01248.x 6. Hatch S, Dohrenwend B. Distribution of traumatic and other stressful life
3. Jensen-Doss A, Hawley KM. Understanding barriers to evidence-based as- events by race/ethnicity, gender, SES and age: A review of the research.
sessment: Clinician attitudes toward standardized assessment tools. J Clin Am J Community Psychol. 2007;40(3-4):313–332.
Child Adolesc Psychol. 2010;39(6):885-896. doi:10.1080/ 7. Johnson J, Hall LH, Berzins K, Baker J, Melling K, Thompson C. Mental
15374416.2010.517169 healthcare staff well-being and burnout: A narrative review of trends,
causes, implications, and recommendations for future interventions. Int J
Ment Health Nurs. 2018;27(1):20-32.
8. Los Angeles Times. Mapping L.A. neighborhoods. 2019; http://
A85 maps.latimes.com/neighborhoods/.
Looking beyond the clinic door: examining the relationship
between clinic neighborhood characteristics and therapist
emotional exhaustion in a large-scale implementation effort A86
Mary Kuckertz1, Anna Lau2, Teresa Lind1, Kenny Le2, Mojdeh Monitoring treatment engagement: how do providers know when
Motamedi1, Lauren Brookman-Frazee1 youth and families are engaged in or disengaged from treatment?
1
University of California, San Diego, La Jolla, CA, USA; 2University of Ellie Wu1, Kimberly Becker1, Anna Hukill1, Bruce Chorpita2
1
California, Los Angeles, Los Angeles, CA, USA University of South Carolina, Columbia, SC, USA; 2University of
Correspondence: Mary Kuckertz ([email protected]) California, Los Angeles, Los Angeles, CA, USA
Implementation Science 2020, 15(Suppl 2):A85 Correspondence: Ellie Wu ([email protected])
Background
In an effort to increase the use of evidence-based practices (EBPs) in Background
community mental health settings, large-scale implementation efforts Poor engagement of youth and families in mental health services is a
are becoming increasingly common, prompting efforts to better under- significant barrier to implementing psychosocial interventions, given
stand determinants of implementation [1-2]. While much attention has that more than 50% of youth drop out of services before the comple-
been given to the inner and outer organizational and leaderships con- tion of treatment. Detecting engagement problems early is a critical
texts that influence implementation efforts, [3-4] there has been less step in preventing premature termination and promoting successful
focus on the links between therapist factors and the community neigh- implementation of psychosocial interventions in community settings.
borhood in which they work. Clients from high risk neighborhoods Despite this, little is known about how providers assess client engage-
have been shown to present with more severe symptoms [5] and an in- ment in therapy in the absence of structured feedback. The current
creased number of emergent life events that can impact the treatment study examines the indicators that providers use to detect engagement
process [6]. This may influence implementation efforts in the form of and disengagement in a variety of community mental health settings.
therapist burnout, which is linked to higher turnover [7]. Materials and Methods
Materials and Methods As a part of a training workshop, 39 mental health providers were asked
The current study utilized survey data from therapists in Los Angeles to report a case example that represented an engagement challenge, a
County collected within the context of a system-driven multiple EBP case example that represented an engagement success, and indicators
implementation effort as well as publicly available data published by used to assess engagement for each case. Participating providers worked
the Los Angeles Times “Mapping L.A.” project [8] (e.g., mean income, primarily in schools, as well as community mental health centers, juvenile
race/ethnicity, home ownership, education etc.). Clinic addresses detention centers, and private practices. Case examples and described in-
were used to identify the neighborhood of the clinic as determined dicators were coded based on five domains of engagement (relationship,
by the “Mapping L.A.” project, and neighborhood characteristics were expectancy, attendance, clarity, and homework).
collected and matched to survey data. Multilevel modeling was used Results
(level 1=therapist, level 2=program, level 3=neighborhood) to exam- Frequencies of coded engagement indicators were examined across en-
ine the relationship between therapist emotional exhaustion and gagement challenge and engagement success case examples. Results
characteristics of the community of the clinic. showed that low attendance was the most commonly reported indicator
Results of disengagement, making up 75% of the indicators coded in the en-
Analyses showed population density (β=.14, p<.05), median income gagement challenge case examples. On the contrary, homework comple-
(β=-.30, p<.05), and home ownership (β=.30, p<.05) of the neighbor- tion was the most commonly reported indicator of positive engagement,
hood surrounding the clinic to be significant predictors of therapist making up 69% of the indicators in the engagement success case exam-
emotional exhaustion. ples. A figure will show the distribution of the other engagement do-
Conclusions mains across coded indicators of engagement and disengagement.
Results can help target specific implementation supports towards Conclusions
clinics with therapists at higher risk of emotional exhaustion, thus po- The purpose of this study was to investigate how providers detect
tentially improving the implementation of EBPs. engagement during treatment. While these results are limited due to
small sample size and reporting single case examples, our study sug-
References gests that providers focus on attendance when assessing client dis-
1. McHugh RK, Barlow DH. The dissemination and implementation of engagement, and homework completion when assessing client
evidence-based psychological treatments. A review of current efforts. Am engagement. These results highlight an opportunity to train pro-
Psychol. 2010;65(2):73–84. viders in understanding the multidimensional nature of engagement,
so that they may address signs of attitudinal disengagement before mental health worker at a convenient, neighborhood hub site. MOMS
they lead to disruptions in attendance and treatment dropout. is in the process of expanding to Bridgeport, Connecticut; New York
City; and Washington D.C. This expansion highlights the interplay of
A87 two common paradigms of program replication – designing program
Lay counselor burnout and turnover across systems: are there components using community-based participatory research methods
differences that are important for implementation of task-sharing and implementing programs with a high degree of fidelity. These
approaches? two goals are often seen as contradictory, but successful programs
Leah Lucid1, Prerna Martin1, Christine L. Gray2, Rosemary are known to appraise both. Our first objective is thus to describe
Meza1, Augustine I. Wasonga3, Kathryn Whetten2, Shannon Dorsey1 MOMS replication through community partners, identifying which
1
University of Washington, Seattle, WA, USA; 2Duke University, Durham, components have been malleable based on community feedback
NC, USA; 3ACE Africa Kenya, Bungoma, Kenya and which were static. Our second objective is to provide the frame-
Correspondence: Leah Lucid ([email protected]) work to measure program fidelity to the original model.
Three case studies are presented: the first two discuss lessons learned
Background from early program development in Bridgeport and New York, begin-
Less than 1% of children in low- and middle-income countries (LMICs) ning with iterative co-design of a needs assessment. The third exam-
with mental healthcare needs will receive treatment, in part due to a ines D.C. MOMS beginning with the needs assessment distributed
dearth of trained providers. Task-sharing, in which lay counselors deliver through the Technical Assistance for Needy Families (TANF) program.
treatment, is an implementation solution; however, more research is Results of a structural fidelity assessment in D.C., which operationalizes
needed on counselors’ experience in this role. Although providing trauma fidelity components including content and process, participant engage-
treatment has been linked to provider burnout in the US, to our know- ment, adherence to goals and needs assessment and theory of change,
ledge no one has examined this question with lay counselors in LMICs. and acceptability of programs, are also provided.
The present study compares burnout and turnover between lay coun- Insights from these case studies span issues of fostering and main-
selors in two systems in Kenya. Lay counselors included 60 teachers and taining positive relationships with community groups, identifying
community health volunteers (CHVs) delivering group-based Trauma- government partners through which replication can be sustainable,
focused Cognitive Behavioral Therapy (TF-CBT) as part of a large NIMH- but still acceptable to community members, managing capacity con-
funded trial. These lay counselors embedded in different systems, with straints particularly in environments that are saturated with nonprofit
differing supports, role definitions, and time demands, may have differing programming, navigating issues of population representation, and
experiences of adding a counseling role to their existing one (as teacher overcoming unforeseen challenges based on context.
or CHV). Immediately after completing TF-CBT training, the lay counselors Conclusions
reported their baseline job burnout and turnover intention for their ori- Centering community nuance while replicating successful mental
ginal role as either a teacher or CHV. Approximately six months later, after health interventions is an essential, iterative, and difficult process. Ex-
leading two rounds of TF-CBT groups, they completed post- periences of the early career investigators at MOMS while replicating
implementation surveys assessing constructs of interest. Analyses used program in three new cities provide reflections on lessons learned
independent samples t-tests to determine if there were significant differ- when entering new communities ethically and effectively, learning
ences between teacher and CHV reports. how to work towards program fidelity while valuing responsiveness
Results to community need.
At baseline, CHVs reported significantly higher burnout in their original
role than teachers (p<.001) but there was no difference in turnover A89
intention. Post-implementation, teachers reported significantly higher Applying a causal model to the implementation of evidence-based
counseling-specific burnout (p=0.005) and compassion fatigue (p=0.007). practice for autistic adults in community settings
They also reported higher job turnover intention (p=0.003) than CHVs. Brenna Maddox, Rinad S. Beidas, Jessica Fishman, Samantha Crabbe,
There were no differences for post-implementation compassion satisfac- David Mandell
tion (p=0.91) or intention to continue their counselor role (p=0.04). University of Pennsylvania, Philadelphia, PA. USA
Conclusions Correspondence: Brenna Maddox ([email protected])
Our findings contribute to the limited literature on the lived experi- Implementation Science 2020, 15(Suppl 2):A89
ence of lay counselors providing mental health treatment to families
in LMICs. Some aspects of experience differ by system, and these dif- Background
ferences may suggest points for intervention to retain lay counselors Cognitive-behavioral therapy (CBT) can improve anxiety and depres-
for scale-up and sustainment or to determine which systems may be sion in autistic adults [1], who frequently struggle with these co-
more viable for “adding” a counseling role. Retaining lay counselors occurring psychiatric conditions [2]. Most autistic adults do not re-
after investing in their training is critical for efforts aimed at reducing ceive CBT, however, because of a lack of clinicians who are willing
the substantial treatment gap in LMICs. and able to treat them. We applied the Theory of Planned Behavior
Trial Registration: Clinicaltrials.gov NCT03243396 (TPB) [3], a leading model of behavior change, to examine malleable
factors that may influence community clinicians’ use of CBT for autis-
A88 tic adults with anxiety or depression. These factors can be targeted
Maintaining community partnership and program fidelity while with tailored implementation strategies to improve implementation
replicating a community-based mental health intervention: a case of evidence-based practice [4].
study of the New Haven MOMS Partnership® replication in Materials and Methods
Bridgeport, Connecticut, New York City, and Washington D.C One hundred clinicians completed an online survey. We used stan-
Sonia Taneja, Megan Smith dardized procedures from social psychology to measure clinicians’ in-
Yale School of Medicine, New Haven, CT, USA tentions, attitudes, norms, and self-efficacy, [5] and adapted them to
Correspondence: Sonia Taneja ([email protected]) focus on using CBT with adult clients (both autistic and non-autistic)
Implementation Science 2020, 15(Suppl 2):A88 who present for anxiety or depression treatment.
Results
Background Clinicians reported weaker intentions (p = .001, d = .34), less favor-
The Mental health Outreach for MotherS (MOMS) Partnership is a able attitudes (p < .001, d = .69), less descriptive normative pressure
community-academic partnership in New Haven responsible for de- (p < .001, d = .39), less injunctive normative pressure (p < .001, d =
veloping a maternal mental health intervention – a cognitive behav- .66), and worse self-efficacy (p < .001, d = .81) to start CBT with autis-
ioral group therapy course co-led by a clinician and a community tic adults than with non-autistic adults. The only significant predictor
of intentions to begin CBT with clients (both autistic and non-autistic) practitioner, organization, and contextual determinants influencing imple-
who present for anxiety or depression treatment was clinicians’ attitudes mentation; local adaptations (what, when, how, why, by whom, where,
(p < .001), with more favorable attitudes predicting stronger intentions. and what impact the modification had). We will also share whether PEAR
Conclusions LS is acceptable, feasible, and appropriate as an intervention to increase
For the purposes of this study, attitudes refer to the clinicians’ per- social connectedness. Local CBO partners will share how partnering with
ceived advantages and disadvantages of starting CBT with their adult implementation scientists has impacted their practice and policymaking,
clients with anxiety or depression. This concept is similar to the im- including facilitators, barriers and opportunities to partnership and appli-
plementation science constructs of “acceptability” and “appropriate- cation of IS.
ness” [4,6]. Knowing that clinicians’ attitudes strongly predicted Conclusions
intentions is valuable for designing effective, tailored implementation Many of the IS frameworks applied in this study have been devel-
strategies to increase clinicians’ adoption of CBT for autistic adults. oped with clinical or public mental health settings. This scaling-out
For example, an implementation strategy targeting attitudes could [9] evaluation will share learnings from researchers and practitioners
include message content to change thinking around the perceived who are applying IS to improve practice and equity [10] for older
fit of CBT with autistic adults. Social psychologists have successfully adults living in poverty.
used this type of approach to change attitudes about complex be-
haviors, [5] but it has not yet been applied to improving the imple- References
mentation of evidence-based practice for autistic adults. 1. Ciechanowski P, Wagner E, Schmaling K, Schwartz S, Williams B, Dierhr P,
Kulzer J, Gray S, Collier C, LoGerfo J. Community-integrated home-based
References depression treatment in older adults: a randomized controlled trial. JAMA
1. Spain D, Sin J, Chalder T, Murphy D, Happé F. Cognitive behaviour (2004) 291(13):1569–77.
therapy for adults with autism spectrum disorders and psychiatric co- 2. Holt-Lunstad J. The potential public health relevance of social isolation
morbidity: a review. Res Autism Spect Dis. 2015;9(1):151-162. and loneliness: prevalence, epidemiology, and risk factors. Public Policy
2. Buck TR, Viskochil J, Farley M, Coon H, McMahon WM, Morgan J, Bilder Aging Rep. 2017;27(4):127-130.
DA. Psychiatric comorbidity and medication use in adults with autism 3. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the
spectrum disorder. J Autism Dev Disord. 2014;44(12):3063-3071. framework method for the analysis of qualitative data in multi-
3. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis disciplinary health research. BMC Med Res Methodol. 2013;13(1):117.
Process. 1991;50(2):179-211. 4. Parkinson S, Eatough V, Holmes J, Stapley E, Midgley N. Framework
4. Fishman JM, Lushin V, Lawson G, et al. A theory for implementation analysis: a worked example of a study exploring young people’s
prediction (TIP): applying causal models of behavior change to specific experiences of depression. Qual Res Psychol. 2016;13(2):109-129.
evidence-based practices. Manuscript under review. 5. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC,
5. Fishbein M, Ajzen I. Predicting and changing behavior: the reasoned Glass JE, York JL. A compilation of strategies for implementing clinical
action approach. New York, NY: Psychology Press; 2010. innovations in health and mental health. Med Care Res Rev.
6. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, 2012;69(2):123-157.
Griffey R, Hensley M. Outcomes for implementation research: conceptual 6. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework
distinctions, measurement challenges, and research agenda. Adm Policy and coding system for modifications and adaptations of evidence-based
Ment Health. 2011;38(2):65-76. interventions. Implement Sci. 2013;8:65
7. Rabin BA, McCreight M, Battaglia C, Ayele R, Burke RE, Hess PL, Frank JW,
Glasgow RE. Systematic, multimethod assessment of adaptations across
A90 four diverse health systems interventions. Front Public Health. 2018;6:102.
Harnessing implementation science with community-based social 8. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A,
service organizations to address depression and social isolation for Griffey R, Hensley M. Outcomes for implementation research: conceptual
older adults living in poverty distinctions, measurement challenges, and research agenda. Adm Policy
Correspondence: Lesley Steinman ([email protected]) Ment Health. 2011;38(2):65-76. doi:10.1007/s10488-010-0319-7.
University of Washington, Seattle, WA, USA 9. Aarons GA, Sklar M, Mustanski B, Benbow N, Brown CH. “Scaling-out”
Implementation Science 2020, 15(Suppl 2):A90 evidence-based interventions to new populations or new health care de-
livery systems. Implement Sci. 2017;12(1):1-13. doi:10.1186/s13012-017-
Background 0640-6.
Late-life depression (LLD) is a major public health issue that impacts older 10. Chinman M, Woodward EN, Curran GM, Hausmann LRM. Harnessing
adults, families, communities, and health systems. Often unrecognized or implementation science to increase the impact of health disparity
undertreated among older populations, those living in poverty experience research. Med Care. 2017; 55(Suppl 9 2): S16–S23.
disparities in access to LLD care. PEARLS is a home-based collaborative care
model (CCM) for LLD developed with social service community-based orga-
nizations (CBOs) to reach underserved older adults. Since demonstrating ef- A91
fectiveness via RCT in 2004, [1] we have partnered with CBOs to apply Reflections on a decade of policy/practice-driven implementation
implementation science (IS) tools and strategies to improve PEARLS delivery. research: strategies for meaningful collaboration
One of our current projects is to evaluate whether and how PEARLS may Correspondence: Sarah Kaye ([email protected])
help address the recent “epidemic” of social isolation [2] in five US sites. Kaye Implementation & Evaluation, LLC, Washington, DC, USA
Materials and Methods Implementation Science 2020, 15(Suppl 2):A91
The PEARLS Connect Study is a concurrent mixed methods evaluation. Im-
plementation data will be collected via surveys and semi-structured inter- Background
views with 10 CBO administrators and 30 PEARLS practitioners. The This presentation aims to discuss three questions of interest to the
interview guide asks about implementation strategies, determinants, adapta- Society for Implementation Research Collaboration (SIRC) at the 2019
tions, and outcomes. We will use framework analysis [3-4] to map text data conference: (1) What can implementation researchers, practitioners
to a priori domains from IS frameworks – ERIC implementation strategies, [5] and intermediaries learn from each other? (2) How can we adapt les-
EPIS implementation determinants (Aarons), QUERI Adapted Stirman [6] sons learned from implementation science in ways that are culturally
Adaptation Framework (QASAF) [7] and implementation outcomes [8]. We and contextually appropriate? (3) What are examples of strategies for
will also use inductive thematic analysis to pull out any other key implemen- collaboration between policy makers/funders and implementation
tation themes that emerge from the conversations with practitioners. experts? Two-way communication between the research world and
Results the real world is vital to minimizing the gap between research and
Data are currently being collected (Spring 2019) and will be analyzed in practice. Understanding the needs of policy makers and practitioners
Summer 2019. We will report on implementation strategies (planning, is critical to producing research findings that are relevant to the
educating, financing, restructuring, quality management, policy); users that studies are intended to support. Moreover, research
projects benefit from acknowledging policy, practice, and community organizational stakeholders similar to potential research participants.
expertise–and recognizing community members’ meaningful contri- OAB members reviewed draft intervention materials and provided
butions to a better understanding of implementation strategies, pro- feedback on the structure and content of the COAST-IS intervention.
cesses, and causal mechanisms. They continue to influence intervention material development. To in-
Materials and Methods corporate the perspectives of families and youth during intervention
Drawing on principles from community-engaged research [1] and re- development, NCCTS leadership connected the research team with
search/evaluation capacity-building [2], this presentation offers strat- two existing client groups. The Family and Youth Insight Advisory
egies that implementation researchers might consider when Group and the Youth Task Force met with the research team to dis-
partnering with communities. cuss barriers to their engagement in trauma-focused treatments and
Results recommend strategies to address these barriers. The research team
Examples of these strategies include: Co-designing a theory of synthesized these recommendations to share with future interven-
change for the intervention(s) and implementation; Utilizing sequen- tion participants to promote client-focused implementation.
tial explanatory research designs [3] through a process of read, listen, Conclusions
share, discuss, adapt; Offering community empowerment opportun- This study illustrates the potential impact of engaging diverse stake-
ities through voice and choice about design, measures, and theories/ holders in the development of implementation interventions. The
frameworks; Addressing questions and requirements that are critical study team will continue engagement efforts during intervention
to policy makers and practitioners; Identifying dissemination strat- testing, planning for future research, and dissemination.
egies that meet the needs of communities. Trial Registration: Clinicaltrials.gov NCT03799432
1. Wallerstein N, Duran B. Community-based participatory research contri- 1. Chambers DA, Azrin ST. Research and services partnerships: partnership: a
butions to intervention research: the intersection of science and practice fundamental component of dissemination and implementation research.
to improve health equity. Am J Public Health. 2010;100 (Suppl 1):S40-6. Psychiatr Serv. 2013;64(6):509-511.
2. Labin SN, Duffy JL, Meyers DC. A research synthesis of the evaluation 2. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK,
capacity building literature. Am J Eval. 2012;33(3):307–338. https:// Mandell DS. Methods to Improve the Selection and Tailoring of
doi.org/10.1177/1098214011434608. Implementation Strategies. J Behav Health Serv Res. 2017 Apr;44(2):177-
3. Creswell JW, Clark VLP. Designing and conducting mixed methods 194. doi: 10.1007/s11414-015-9475-6.
research studies, Second Ed. SAGE Publications. Thousand Oaks, CA; 2010. 3. Eldredge LKB, Markham CM, Ruiter RA, Kok G, Fernandez ME, Parcel GS.
Planning health promotion programs: an intervention mapping
approach. Hoboken, NJ: John Wiley & Sons;2016.
A92
Engaging stakeholders in the development of an intervention to
systematically tailor implementation strategies A93
Amber Haley1, Sheila Patel1, Jamie Guillergan1, Lisa Amaya Building capacity in advance care planning: an example of
Jackson2, Mellicent Blythe3, Beverly Glienke3, Alicia Sellers4, Jennifer collaboration amongst policy-makers, care providers, community
Grady4, Byron J. Powell5 organizations, and implementation researchers
1
University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; 2Duke Correspondence: Robin Urquhart ([email protected])
University, Durham, NC, USA; 3North Carolina Child Treatment Program, Dalhousie University, Halifax, Nova Scotia, Canada
Durham, NC, USA; 4National Center for Child Traumatic Stress, Durham, Implementation Science 2020, 15(Suppl 2):A93
NC, USA; 5Washington University in St. Louis, St. Louis, MO, USA
Correspondence: Amber Haley ([email protected]) Background
Implementation Science 2020, 15(Suppl 2):A92 Advance care planning (ACP) is the process by which patients, along-
side their healthcare providers, consider options about future health-
Background care decisions. Although ACP is associated with improved outcomes
Stakeholder engagement is often considered critical to implementa- as people near end-of-life [1-3], many providers report discomfort
tion science [1]. This study illustrates the value of engaging a wide with ACP and subsequent goals-of-care (GOC) conversations [4].
variety of stakeholders in the development of an implementation There are also many organizational, community, and system barriers
intervention. This project was funded by the National Institutes of to these conversations [5-6]. Changing practice, and ultimately im-
Mental Health to develop and pilot the Collaborative Organizational proving patient/family outcomes, will require collaborative efforts
Approach to Selecting and Tailoring Implementation Strategies from multi-level/sector partners. Through collaboration amongst gov-
(COAST-IS). ernment, the provincial cancer agency, care providers, a community-
Materials and Methods based organization, and an implementation scientist, we sought to
The COAST-IS intervention will equip organizations to use Interven- develop provider capacity in initiating, and increase the frequency of,
tion Mapping to select and tailor implementation strategies to ad- ACP/GOC conversations with cancer patients.
dress site-specific determinants of treatment implementation and Materials and Methods
sustainment [2]. Intervention Mapping is a multistep process that in- Informed by the knowledge-to-action framework [7] and data gath-
corporates theory, evidence, and stakeholder perspectives to ensure ered from various sources, including formal research studies and
that intervention components effectively address key determinants local context, we co-designed an intervention (communication skills
of change [3]. The first year of the grant focused on engaging na- training workshop, clinician guides and documentation tools, and pa-
tional and local experts, organizational leaders, clinicians, caregivers, tient/family guide). We tested the intervention with 51 providers in
and youth in the development of COAST-IS for a trauma-focused oncology, palliative, and primary care; evaluated it using post-
treatment. The approaches to engagement in this project range from workshop surveys, focus groups, and chart reviews; and adapted it
full partnership to stakeholder consultation. The project is being con- via iterative team dialogue and debriefing.
ducted in partnership with leadership from the UCLA-Duke National Results
Center for Child Traumatic Stress (NCCTS) and the North Carolina Data from the first two (of five) workshops revealed that most pro-
Child Treatment Program (NC CTP). NCCTS and NC CTP continue to viders were uncomfortable with ACP/GOC conversations and it was
shape project planning and intervention development through con- premature, at that point, to expect them to complete documenta-
tinuous feedback on intervention structure, organizational assess- tion. Thus, we focused on refining/enhancing the workshop and en-
ment, and dissemination planning. couraging use of the clinician/patient guides. Our findings
Results demonstrated increased provider confidence across most ACP/GOC
The research team worked with NC CTP to convene an domains and an increased number of ACP/GOC conversations with
Organizational Advisory Board (OAB) to solicit the perspectives of patients, and provided critical insight for scale-up (e.g., train-the-
trainer strategies, integration with existing education events). The analyses assessed differences in the perceptions between state mental
adapted intervention was subsequently implemented province-wide. health agency and state insurance agency directors.
Team reflection and debriefing revealed at least 6 factors critical to Results
successful implementation: 1. underpinned by research and local evi- A significantly higher proportion of state mental health agency direc-
dence; 2. driven collaboratively by multi-level/sector stakeholders; 3. tors thought that there would be a benefit to inter-agency collabor-
guided by a plan but a willingness to adapt as needed; 4. ongoing ation than state insurance agency directors (95.4% vs. 67.7%, χ2 p=
evaluation; 5. clinical champions; and 6. a dedicated coordinator to .001). A significantly higher proportion of state mental health agency
bring to all together. directors identified different agency culture (e.g., “norms, values”)
Conclusions (51.2% vs. 24.2%, χ2 p= .017) and different agency terminology
Collaboration amongst multi-level/sector stakeholders has the poten- (51.2% vs. 18.2%, χ2 p=.003) as major barriers to inter-agency collab-
tial to change clinical practice. Evidence from multiple sources is crit- oration than state insurance agency directors. For all five barriers to
ical to designing an evidence-based, locally-adapted intervention inter-agency collaboration, the proportion of respondents identifying
and convincing a myriad of stakeholders to support it. each as a major barrier was higher among state mental health
agency directors that state insurance agency directors.
References Conclusions
1. Detering KM, Hancock AD, Reade MC, Silvester W. The impact of advance State mental health agency directors believe that there is more bene-
care planning on end of life care in elderly patients: randomised fit to inter-agency collaboration related to the implementation of
controlled trial. BMJ 2010;340:c1345 federal mental health parity policy than state insurance agency direc-
2. Lautrette A, Darmon M, Megarbane B, Joly LM, Chevret S, Adrie C, tors, but also believe that there are more barriers to collaboration.
Barnoud D, Bleichner G, Bruel C, Choukroun G, Curtis JR, Fieux F, Galliot Implementation strategies such as consensus discussions, facilitation
R, Garrouste-Orgeas M, Georges H, Goldgran-Toledano D, Jourdain M, between state mental health agencies and state insurance agencies,
Loubert G, Reignier J, Saidi F, Souweine B, Vincent F, Barnes NK, Pochard and the development of an implementation glossary could improve
F, Schlemmer B, Azoulay E. A communication strategy and brochure for inter-agency collaboration and enhance the implementation of fed-
relatives of patients dying in the ICU. N Engl J Med 2007;356:469-78 eral mental health parity policy [2-3].
3. Wright AA, Zhang B, Ray A, Mack JW, Trice E, Balboni T, Mitchell SL,
Jackson VA, Block SD, Maciejewski PK, Prigerson HG. Associations References
between end-of-life discussions, patient mental health, medical care near 1. The Kennedy Forum. Evaluating state mental health and addiction parity
death, and caregiver bereavement adjustment. JAMA 2008;300:1665-73 statutes: a technical report. 2018. https://chp-wp-
4. Morrison RS, Morrison EW, Glickman DF. Physician reluctance to discuss uploads.s3.amazonaws.com/www.paritytrack.org/uploads/2018/09/KF-
advance directives. An empiric investigation of potential barriers. Arch Evaluating-State-Mental-Health-Report-0918_web.pdf
Intern Med 1994;154:2311-8 2. Purtle J, Borchers B, Clement T, Mauri A. Inter-agency strategies used by
5. Batchelor F, Hwang K, Haralambous B, Fearn M, Mackell P, Nolte L, state mental health agencies to assist with federal behavioral health par-
Detering K. Facilitators and barriers to advance care planning ity implementation. J Behav Health Serv Res. 2018; 45(3):516-526.
implementation in Australian aged care settings: a systematic review and 3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM,
thematic analysis. Australas J Ageing. 2019;38(3):173-181. doi: 10.1111/ Proctor EK, Kirchner JE. A refined compilation of implementation strategies:
ajag.12639. results from the Expert Recommendations for Implementing Change (ERIC)
6. Howard M, Bernard C, Klein D, Elston D, Tan A, Slaven M, Barwich D, You project. Implement Sci. 2015;10(1):21. doi: 10.1186/s13012-015-0209-1
JJ, Heyland DK. Barriers to and enablers of advance care planning with
patients in primary care: survey of health care providers. Can Fam
Physician. 2018; 64(4):e190-e198 A95
7. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, A rapid review to inform implementation of a behavioral health
Robinson N. Lost in knowledge translation: time for a map? J Contin intervention in primary care: methods and outcomes
Educ Health Prof. 2006;26(1):13-24 Madeline Larson, Mimi Choy-Brown, Scott Marsalis
University of Minnesota, Minneapolis, MN, USA
Correspondence: Madeline Larson ([email protected])
Comparing state mental health agency and state insurance agency
directors’ perspectives on the benefits and barriers to inter-agency Background
collaboration related to implementation of federal mental health Current methods or approaches used in scientific practice have failed
parity policy to rapidly and rigorously integrate cultivated knowledge into feasible
Katherine Nelson, Jonathan Purtle and sustainable real-world practices and policies [1]. Continuing to
Drexel University, Philadelphia, PA, USA use traditional scientific approaches runs the risk of implementation
Correspondence: Katherine Nelson ([email protected]) science falling short of expectations [1]. Different methods are
Implementation Science 2020, 15(Suppl 2):A94 needed to foster the rapid integration of science and practice while
maintaining rigor [1]. The purpose of this study is to conduct a rapid
Background review of the literature to identify implementation determinants and
There is substantial state-level variation in the implementation of federal strategies that impact adoption, implementation, and sustainability
mental health parity policy—which requires that insurance companies of family-based mental health interventions in primary care.
provided equal coverage for mental and physical health care [1]. One Materials and Methods
possible reason for this variation could be differences in how state men- A rapid review will be conducted that involves a primary literature
tal health and insurance agencies are collaborating to support implemen- search of Medline, Embase, PsycInfo, CINAHL databases to identify
tation [2]. This study aimed to: characterize perceptions of the benefits existing implementation strategies used to foster the uptake and use
and barriers to inter-agency collaboration related to implementation of of family-based interventions in primary care settings. A secondary
federal parity policy, and compare how these perceptions differ between search will be performed as an iterative process and included biblio-
state mental health agency and state insurance agency directors. graphic and grey literature searches of reference lists, authors and
Materials and Methods specifically the journal, Implementation Science. A systematic ap-
Web-based surveys of state mental health agency directors (n=43, re- proach to data extraction will be used to illuminate key determi-
sponse rate= 84%) and state insurance agency directors (n=34, re- nants, strategies, and outcomes related to implementation.
sponse rate= 67%) were conducted in 2017. One item assessed the Results
perceived benefit of collaboration between two the agencies and five The results of the rapid review will be presented along with method
items assessed specific barriers to collaboration. All items were explicitly parameters (e.g., time to complete, cost and resources, streamlining
about implementation of federal mental health parity policy. Bivariate processes). Findings will directly inform an actionable plan to
implement the intervention in primary care settings. Results will be References

used to inform stakeholder decisions regarding implementation sup- 1. Campbell JA, Walker RJ, Egede LE. Associations between adverse
ports provided during a multi-site rollout of FBCI in integrated pri- childhood experiences, high-risk behaviors, and morbidity in adulthood.
mary care settings throughout the USA. Am J Prev Med. 2016;50(3):344-352.
Conclusions 2. Talbot JA, Szlosek M, Ziller EC. Adverse childhood experiences in rural
Rapid reviews can be used to integrate implementation research into and urban contexts. University of Southern Maine, Muskie School of
practice or policy. While it is not exhaustive, rapid reviews provide a Public Service, Maine Rural Health Research Center. 2016. PB-64.
pragmatic and systematic approach to synthesizing evidence to in- 3. Brown JD, King MA, Wissow LS. The central role of relationships with
form decision-making in real-world efforts. Future work will compare trauma-informed integrated care for children and youth. Acad Pediatr.
results of rapid review to results of a gold-standard systematic 2017;17(7):S94-S101.
review. 4. Hummer VL, Dollard N, Robst J, Armstrong MI. Innovations in
implementation of trauma-informed care practices in youth residential
Reference treatment: a curriculum for organizational change. Child Welfare. 2010;
1. Glasgow RE, Chambers D. Developing robust, sustainable, 89(2):79.
implementation systems using rigorous, rapid and relevant science. Clin 5. Airhihenbuwa CO, Shisana O, Zungu N, BeLue R, Makofani DM, Shefer T,
Transl Sci. 2012;5(1):48-55. Smith E, Simbayi L. Research capacity building: a US-South African part-
nership. Glob Health Promot. 2011;18(2):27-35.
6. Chavis DM. Building community capacity to prevent violence through
A96 coalitions and partnerships. J Health Care Poor Underserved.
Rural primary care organizational change toward trauma-informed 1995;6(2):234-245.
integrated primary care through community partnerships
Deborah Moon1, Eve-Lynn Nelson2, Michelle Johnson-
Motoyama3, Shawna Wright4, Becci Akin4 A97
1
University of Pittsburgh, Pittsburgh, PA, USA; 2University of Kansas The impact of new national and state insurance policy on
Medical Center, Kansas City, KS, USA; 3University of Ohio implementation of the collaborative care model for perinatal
State, Columbus, OH, USA; 4University of Kansas, Kansas City, KS, USA depression: the rubber hits the road for government policy to
Correspondence: Deborah Moon ([email protected]) support behavioral health integration
Implementation Science 2020, 15(Suppl 2):A96 Ian Bennett1, Ashok Reddy2,3, Anna Ratzliff1, Stephanie Shushman4, Jay
Wellington5
1
Background University of Washington, Seattle, WA, USA; 2Centers for Medicare and
Rural children are frequently exposed to adverse experiences, which Medicare Services Innovation Center, Baltimore, MD, USA; 3VA Puget
are associated with negative long-term health outcomes [1-2]. Trauma- Sound Medical Center, Seattle, WA, USA; 4Community Health Plan of
informed integrated primary care provides a model of care that takes Washington, Seattle, WA, USA; 5UW Medicine Neighborhood Clinics,
into account the impact of social determinants of health both at the Seattle, WA, USA
prevention and treatment levels [3]. Developing into trauma-informed Correspondence: Ian Bennett ([email protected])
integrated primary care involves complex organizational change pro- Implementation Science 2020, 15(Suppl 2):A97
cesses that are challenging for rural primary care facilities burdened
with multiple priorities [4]. Community partnership presents capacity Background
building opportunities for rural primary healthcare clinics in such pro- The collaborative care model (CoCM), is a highly evidence based
cesses [5-6]. The purpose of this study was to examine facilitators and complex intervention for management of common mental disorders
barriers in rural primary care organizational changes toward Trauma- in primary care. Despite decades of effort to disseminate and imple-
Informed Integrated Primary Care based on theoretical frameworks of ment this model penetration into clinical practice is modest. A major
organizational change and implementation science, with an emphasis obstacle is the lack of reimbursement for team care activities. In
on partnering as a key process of the organizational change efforts. January 2018, a new mechanism for payment of the work of a CoCM
Materials and Methods team (G and CPT billing codes) was introduced by the US Centers for
This study was a community engaged organizational case study funded Medicare & Medicaid Services for Medicare recipients. Washington
by HRSA, Doris Duke Foundation, and NY Community Trust Fund. The State has extended this mechanism to Medicaid recipients (Managed
study was conducted in collaboration with a Federally Qualified Health Care and fee for service). Perinatal depression (depression in preg-
Center in a rural community of a midwestern state, a non-profit nancy and the year postpartum), is the most common clinical mater-
organization, and an academic research and development center. Data nal disorder. Untreated depression is associated with poor persistent
collection involved key informant interviews, surveys, direct observation developmental, learning, and mental health outcomes for children.
of formal and informal interactions with key stakeholders. Data analyses Medicaid funds approximately 50% of perinatal care making the new
focused on identifying facilitating and/or interfering contexts and CoCM codes a viable means of funding care for perinatal depression
mechanisms in the organizational change efforts. This poster focuses in this vulnerable population.
on contexts and mechanisms pertaining to the partnering aspect. Materials and Methods
Results MInD-I (maternal infant dyad-implementation), is an NIMH funded
Facilitating contexts included shared mission and values, commitment to (R01MH108548) implementation trial of CoCM for perinatal depres-
learning and innovation, increased awareness of social determinants of sion with five sites in Washington and thirteen outside.
health among leadership, core work group, and transparent communica- Results
tion. The key facilitating mechanisms were leveraged resources and Despite the availability of CoCM CPT codes to support the work of team
shared expertise that positively influenced change readiness. Major inter- care for depression they have not been widely utilized even in settings
fering contexts included limited understanding of the workflow, adaptabil- with collaborative care already in place. The UW Neighborhood Clinic
ity, and performance measurement. The key interfering mechanism was primary care network undertook steps needed to document and
the culture clash that negatively influenced engaging key stakeholders. process these codes for billing. The availability of these billing codes
Conclusions was used to support implementation of CoCM for perinatal depression
Partnering is a key process in rural primary care capacity building to within four sites of this network. New administrative, work flow, and
meet the complex healthcare needs of children in under-resourced interdisciplinary elements of this care represented challenges.
communities through trauma-informed integrated care. Understand- Conclusions
ing facilitating and interfering contexts and mechanisms in partner- Members of the panel will describe the national and state health pol-
ing can inform the process of building strategic partnerships for icy goals underlying the promulgation of these codes and the
collective actions toward building a healthy community in under- current rate of utilization of these codes. The impact of these codes
served regions. on implementation of CoCM for perinatal depression will be explored
from the health system and primary care clinic level provider per- (v1) of 93 hierarchically clustered techniques: building an international
spectives. The panel will provide a unique multi-level perspective on consensus for the reporting of behavior change interventions. Ann
the impact of a novel funding policy on the implementation of a Behav Med. 2013; 46(1):81-95.
highly evidence based complex intervention requiring change in
practice organization. A99
Unpacking and re-packing what we know about barriers and
A98 facilitators assessments
Active ingredients of implementation: examining the overlap Sobia Khan1, Julia Moore1, Byron J. Powell2
1
between behaviour change techniques and implementation The Center for Implementation, Toronto, Ontario, Canada; 2Washington
strategies University in St. Louis, St. Louis, MO, USA
Sheena McHugh1, Justin Presseau2, Courtney Leucking3, Byron J. Powell4 Correspondence: Sobia Khan (sobia.khan@thecenterforimplementation.
1
University College Cork, Cork, Ireland; 2Ottawa Hospital Research com)
Institute, Ottawa, Ontario, Canada; 3University of North Carolina at Implementation Science 2020, 15(Suppl 2):A99
Chapel Hill, Chapel Hill, NC, USA; 4Washington University in St. Louis, St.
Louis, MO, USA Background
Correspondence: Sheena McHugh ([email protected]) Barriers and facilitators assessments (BFAs) are regarded as a key step
Implementation Science 2020, 15(Suppl 2):A98 in implementation, yet our understanding of how to conduct BFAs
effectively has stalled. There is an abundance of literature on barriers
Background and facilitators (BFs) to adopting a multitude of interventions and
Efforts to generate an evidence base for implementation strategies programs, but these studies do not offer any additional insight on
are frustrated by insufficient description [1]. The ERIC compilation how BFAs can be made more meaningful and rigorous. We argue
names and defines implementation strategies [2]; however, further that BFAs should be reconsidered in order to close important gaps in
work is required to operationalise strategies to clearly describe the how barriers and facilitators impact intervention implementation.
specific actions involved [1]. The purpose of this project is to exam- Materials and Methods
ine the extent to which strategies can be specified according to be- We synthesized current approaches to conducting BFAs, and offered
haviour change techniques [3], ‘active ingredients’ of interventions alternate methods and considerations for BFAs.
with the potential to change behaviour. Results
Materials and Methods BFAs published in the literature typically either report lists of BFs
The primary data source was the definitions of 73 strategies con- without describing specific relationships between BFs and implemen-
tained in the ERIC compilation [2]. The definition of each strategy tation or clinical outcomes, or they report how the BFA contributed
was deductively coded using the BCT Taxonomy, [3] which contains to program development, planning, adaptation, or evaluation (e.g.,
93 discrete techniques with the potential to change behaviour. A BFs were used to select strategies to implement the program). How-
typology was developed iteratively to categorise the extent of over- ever, these studies have substantial limitations and do little to better
lap between strategies and BCTs. Three implementation scientists in- interpret BFs, which is necessary to adapt programs and/or systemat-
dependently rated their level of agreement with and confidence in ically develop implementation strategies. First, they are typically con-
the categorisation. ducted before or after implementation, and do not explore the
Results relationship between anticipated versus actual/experienced BFs. Sec-
During preliminary analysis, 86 BCTs were linked to 73 strategies. Five ond, they rely primarily upon self-report, although individuals tend to
types of overlap were identified. 1) In 6 instances, there was a direct have a poor understanding of their own behavior. Third, they fail to
overlap between strategies and BCTs (e.g., strategy: remind clinicians, examine dependencies between BFs (e.g., removing one barrier may
BCT: prompts and cues). 2) In 36 instances, there was at least 1 BCT uncover unintended consequences or “masked” barriers). BFAs would
clearly subsumed under the strategy description which could be be enhanced by: 1) continuous assessment throughout implementa-
used to guide initial operationalisation (e.g., strategy: clinical supervi- tion process; 2) considering their proximity to, and impact on, imple-
sion, BCT: restructure social environment). 3) In 42 instances, a BCT(s) mentation; 3) integrating prioritization processes; 4) increased use of
was probably subsumed under the strategy given its definition and/ observation, qualitative, and mixed methods; and 5) integrating sys-
or title, but other BCTs were possible depending on how the strategy tems science methods that facilitate exploration of
was operationalised (e.g., strategy: visit other implementation sites, interdependencies.
BCT: social comparison). For 8 strategies, there were no BCTs clearly Conclusions
indicated in the strategy definition or title (e.g., strategy: make train- Given the centrality of BFAs in implementation science, it is critical
ing dynamic). Finally, 14 strategies did not focus on behaviour that we continue to examine how they can be improved methodo-
change to support implementation (e.g., strategy: access new logically and how they might yield more accurate and actionable
funding). data. This study advances the field by presenting concrete sugges-
Conclusions tions for how BFAs could be improved.
Many implementation strategies rely on assumptions and inference
on the part of the intervention developer, be it researcher or practi- A100
tioner, to apply them in a setting. This study is the first step towards Demonstrating the value of coincidence analysis for identifying
moving from general descriptions of implementation strategies to successful implementation strategies
full and consistent descriptions of their active ingredients. This is es- Sarah Birken1, Soohyun Hwang1, Laura Viera2, Emily Haines1, Tamara
sential to understanding the mechanisms by which implementation Huson1, Rebecca Whitaker1, Lawrence Shulman3, Deborah Mayer1
1
strategies exert their effects. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; 2North
Carolina Translational and Clinical Sciences Institute, University of North
References Carolina at Chapel Hill, Chapel Hill, NC, USA; 3Commission on Cancer,
1. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: University of Pennsylvania, Philadelphia, PA, USA
recommendations for specifying and reporting. Implementation Science. Correspondence: Sarah Birken ([email protected])
2013;8(1):139. Implementation Science 2020, 15(Suppl 2):A100
2. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
MM, Proctor EK, Kirchner JE. A refined compilation of implementation Background
strategies: results from the Expert Recommendations for Implementing Coincidence analysis (CNA) is a configurational comparative method
Change (ERIC) project. Implement Sci. 2015;10(1):21. similar to qualitative comparative analysis, designed for causal infer-
3. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, ence when combinations of co-occurring conditions determine out-
Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy comes and multiple paths to one outcome may exist – as is often
the case in implementation research. To demonstrate its usefulness based services for ASD and the effect on implementation processes, a
in implementation research, we will use CNA to identify the strat- survey including the Implementation Climate Scale (ICS) [4] and the Im-
egies that cancer programs have used to develop comprehensive ap- plementation Leadership Scale (ILS) [5] was distributed to participants.
proaches to survivorship care plan implementation. The Commission Results
on Cancer (CoC) requires cancer care providers to develop and de- Implementation climate and implementation leadership varied by
liver survivorship care plans (SCPs) to survivors and their primary care profession. For the Selection for Openness domain on the ICS, Mental
providers (PCPs). Cancer programs’ approaches to implementing health providers (B=-2.28, p=.022), Mid-level specialists (B=-2.09,
SCPs in practice substantially vary, ranging from cursory (i.e., devel- p=.020), and Teachers/DSP (B=-2.55, p=.004) all reported lower rat-
oping SCPs to meet requirements without delivering them to survi- ings than High-level administrators. For the Educational Supports do-
vors or PCPs) to comprehensive (i.e., promoting adherence to main on the ICS, Mental health providers (B=-2.08, p=.035), Teachers/
screening and health behavior guidelines and recommended DSP (B=-2.16, p=.010), and School-site principals/administrators (B=-
utilization of follow-up care). 2.70, p=.029) all reported lower ratings than High-level administra-
Materials and Methods tors, while Teachers/DSP also reported lower ratings than Mid-level
We have characterized cancer programs’ approaches to implement- specialists (B=-.99, p=.018). For the Proactive Leadership domain on
ing SCPs using semi-structured telephone interviews with providers/ the ILS, Mental health providers (B=-2.39, p=.003), Mid-level special-
staff in 48 CoC-accredited cancer programs. We are using template ists (B=-2.12, p=.002), and Teachers/DSP (B=-1.64, p=.016) all re-
analysis, combining a priori and emergent codes, and calibrating ported lower ratings than High-level administrators.
qualitative data for CNA purposes (e.g., the presence of a formal sur- Conclusion
vivorship implementation committee). By the time of the conference, Implementation leadership and implementation climate vary across
we will have used CNA to identify strategies that are unique to can- participants suggesting variability more broadly. Implications for im-
cer programs with comprehensive approaches to implementing plementation and sustainment of EBPs in school-based services will
SCPs, using cancer programs with cursory approaches as a compari- be discussed.
son group. The outcome of CNA will be a parsimonious list of strat-
egies that are associated with comprehensive approaches to SCP References
implementation. 1. Annual Disability Statistics Compendium. Special education-students ages
Results 14-21 served under IDEA, Part B, left school, by reason. 2014. http://dis-
Preliminary qualitative analyses suggest that programs varied in their abilitycompendium.org/compendium-statistics/special-education.
approaches to SCP implementation. Key variables included human 2. Spillane JP, Healey K. Conceptualizing school leadership and
and financial resources and infrastructure, including standing com- management from a distributed perspective: An exploration of some
mittees and mechanisms of communication and garnering leadership study operations and measures. Elem School J. 2010;111(2):253-81.
support and provider buy-in. We will present CNA results at the 3. Spillane J. Distributed Leadership. San Francisco, CA: Jossey-Bass; 2006.
conference. 4. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational
Conclusions context for EBP implementation: The development and validity testing of
CNA is a promising method for implementation research, where out- the Implementation Climate Scale (ICS). Implement Sci. 2014;9(1).
comes may be explained by combinations of co-occurring conditions, doi:10.1186/s13012-014-0157-15.
and when multiple paths to one outcome may exist. CNA findings 5. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership
are policy-relevant since they identify multiple combinations of strat- scale (ILS): Development of a brief measure of unit level implementation
egies that diverse organizations may deem appropriate; findings leadership. Implement Sci. 2014;9(1). doi:10.1186/1748-5908-9-45.
from this study will inform more than 1500 CoC-accredited cancer
programs with recommended SCP implementation strategies.
A102
A survey of supervisors’ and managers’ practices and needs to
A101 support evidence-based practice implementation in large
Exploring variability in implementation leadership and climate behavioral health care system in New York state
across organizational level Sapana Patel1,2, Andrea Cole2, Paul Margolies1,2, Nancy Covell1,2, Amy
Melina Melgarejo, Jessica Suhrheinrich Anderson-Winchell3, Lisa Dixon1,2
1
San Diego State University, San Diego, CA, USA Columbia University, New York, NY, USA; 2The New York State
Correspondence: Melina Melgarejo ([email protected]) Psychiatric Institute, New York, NY, USA; 3Access Supports for Living,
Implementation Science 2020, 15(Suppl 2):A101 Middletown, NY, USA
Correspondence: Sapana Patel ([email protected])
Nationwide, 576,000 students were served for Autism Spectrum Dis-
order (ASD) during the 2014-15 school year, an increase of 51% from Background
2007-08 [1]. Given the significant increase in demand for educational Managers and supervisors are the lynchpin of success to practice
services for students with ASD, there is urgent demand to improve im- change within any organization. In a time when behavioral health or-
plementation and sustainment of evidence-based practices (EBP) in ganizations are being asked to shift culture, practice and delivery of
school settings. However, the organizational and leadership structure care from volume to value, manager and supervisor roles are moving
for school-based services for ASD is complex and involves a team of towards ensuring quality of evidence-based practice (EBP) implemen-
providers to account for the complexity of care needed [2-3]. tation. We developed a survey to assess supervisor and manager ex-
Organizational culture and leadership have been found to impact EBP perience with and needed supports for EBP implementation in
use in a variety of settings, but less is known about their impact in community mental health agencies.
schools. As a first step toward tailoring implementation intervention for Materials and Methods
this context, the current proposal explores implementation leadership We created surveys using the implementation science literature [1-3]
and implementation climate in relationship with provider factors. and guidance provided by the Center for Practice Innovations [4]
Materials and Methods provider advisory committee. In March 2019, behavioral health
Participants were 340 school-based providers and administrators who are agency leadership (N=4) across New York State invited their man-
involved in supporting students with ASD. Participants included 19 High- agers and supervisors to take part in the survey. Both surveys quer-
level administrators (Special Education Directors, District-level Administra- ied level of commitment and preparedness (scored on a Likert scale,
tors), 112 Mid-level specialists (Autism Specialist, Behavior Specialist, Pro- e.g., 1 = not at all committed/prepared to 5 = very committed/pre-
gram Specialist) 15 School-site principals or administrators, 153 Teachers pared) to implement EBPs along with needed tools and supports to
and direct service providers (DSP) and 33 Mental health providers (school implement EPBs to fidelity. Surveys also included questions about
psychologist, MFT). To explore the leadership structure within school- EBP topics for additional training.
Results therapists (p<.001). For AIM-HI, therapist role (Staff vs. Trainee) mod-
Data collection will finish in June 2019. Of the 23 survey respondents erated changes in ASD (F(1,154)=4.72, p<.05), KNOW (F(1,154)=11.19,
(supervisors: n of 13; managers n of 10) half are social workers (n=12). p<.01), and CONF (F(1,154)=6.60, p=.01); trainees reported greater in-
Supervisors report that, although they are highly committed to EBP im- creases than staff. Therapists’ perceived ASD expertise moderated
plementation (M = 4.42, SD = 1.11), they are only moderately prepared changes in ASD (F(1,154)=11.00, p<.001) and KNOW (F(1,154)=16.29,
to implement EBPs (M = 3.92, SD = 1.11). Needed resources included p< .001); therapists who did not consider themselves ASD specialists
example case conceptualizations, and workbooks to use with con- reported greater increases than their counterparts. Ethnicity moder-
sumers. Managers’ reports were similar: highly committed to EBP imple- ated changes in CONF (F(1,154)=7.73, p<.05); minority therapists
mentation (M = 4.80, SD = .40) and moderately prepared to implement made more gains compared to White therapists. For usual care thera-
EBPs (M = 4.10, SD = .83). Needed resources include worksheets for role pists, role moderated changes in ASD K&C (F(1,154)=7.4, p<.05); staff
modeling and train the trainers to support EBPs. Managers and supervi- therapists reported increases in ASD knowledge, while trainees re-
sors identified trauma informed care and shared decision making as ported decreases.
EBP topics that they need more training in. Conclusions
Conclusions Therapists who received AIM-HI training reported greater changes in
Managers and supervisors’ commitment to EBP implementation is K&C compared to those in usual care. Additionally, for those in AIM-
necessary, but far from sufficient. Managers and supervisors report HI, therapist’s cultural (ethnicity) and professional (role and ASD ex-
feeling only moderately prepared to implement EBP and would pertise) characteristics moderated the effect of training on K&C. Next
benefit from training focusing on their roles in implementation, as steps include combining results with qualitative data to inform adap-
well as tools and resources designed to help them. tations to improve intervention fit and maximize outcomes.
*Results from multiple level modeling accounting for the nested
References structure of the data will be reported.
1. Birken SA, DiMartino LD, Kirk MA, Shoou-Yih DL, McClelland M, Albert Trial Registration: Clinicaltrials.gov NCT02416323
NM. Elaborating on theory with middle managers’ experience imple-
menting healthcare innovations in practice. Implement Sci. 2016; 11(1):2.
2.Dorsey A104
S, Pullman MD, Kerns SEU, Jungbluth N, Mesa R, Thompson K, Berliner L. The How do you apply implementation science in practice? core
juggling act of supervision in community mental: Implications for supporting competencies for implementation practitioners
evidence-based treatment. Adm Policy Ment Health. 2017;44:838-852. Julia Moore1, Diana Kaan2, Louise Zitzelsberger2, Sobia Khan1
1
3. Dorsey S, Kerns SEU, Lucid L, Pullman MD, Harrison JP, Berliner L, The Center for Implementation, Toronto, Ontario, Canada; 2Health
Thompson K, Deblinger E. Objective coding of content and techniques Canada, Ottawa, Ontario, Canada
in workplace-based supervision of an EBT in public mental health. Imple- Correspondence: Julia Moore (julia.
ment Sci. 2018;13:19. [email protected])
4. Covell NH, Margolies PJ, Myers RW, Ruderman D, Fazio ML, McNabb LM, Implementation Science 2020, 15(Suppl 2):A104
Gurran S, Thorning H, Watkins L, Dixon LB. Scaling up evidence-based be-
havioral health care practices in New York state. Psychiatr Serv. Background
2014;65(6):713-5. doi: 10.1176/appi.ps.201400071. The field of implementation science has advanced in recent years,
but unfortunately this has coincided with a growing divide between
the science and practice of implementation. One strategy to bridge
A103 this gap is training implementation practitioners to apply implemen-
Examining therapist characteristics as moderators of change in tation science to their initiatives in a thoughtful and proactive way.
ASD knowledge and confidence in a hybrid effectiveness/ Effective implementation capacity building should be based on core
implementation trial competencies - the knowledge, skills, attitudes, and behaviors
Kassandra Martinez1, Eliana Hurwich-Reiss2, Lauren Brookman-Frazee2,3 needed to apply implementation science. There is a growing body of
1
SDSU/UCSD Joint Doctoral Program in Clinical Psychology, San Diego, literature on core competencies for implementation scientists, but
CA, USA; 2Child and Adolescent Services Research Center, San Diego, same progress has not been made for core competencies for imple-
CA, USA; 3University of California, San Diego, La Jolla, CA, USA mentation practitioners. Building applied implementation science
Correspondence: Kassandra Martinez ([email protected]) capacity at the practitioner level can foster better implementation
Implementation Science 2020, 15(Suppl 2):A103 and overall improved population-level impacts; therefore, under-
standing the core competencies for applying implementation science
Background at the front line is paramount. The goal of this project was to ex-
Provider attitudes, including knowledge-of and confidence using trapolate and synthesize core competencies for implementation
evidence-based interventions (EBIs), are important outcomes of training practitioners.
interventions and potential mechanisms of EBI delivery. This study uti- Materials and Methods
lized data from a Hybrid Type-1 effectiveness trial of AIM-HI (An Individu- We scanned the published and gray literature to identify core com-
alized Mental Health Intervention for Children with ASD), an intervention petencies for implementation practice. Six documents outlining (or
to reduce challenging behaviors in children with Autism Spectrum Dis- including components of) core competencies for implementation
order (ASD). The following objectives were addressed: 1) examine the ef- practice were retrieved. Two analysts reviewed each document using
fectiveness of AIM-HI training/consultation on changes in therapists’ a content analysis approach. Competencies relevant to implementa-
perceived knowledge and confidence (K&C) of ASD strategies, and 2) tion practice were extracted into an abstraction form and consoli-
examine therapist demographic and professional characteristics as mod- dated into a list of common competencies. The refined list of
erators of training effects. competencies was then grouped thematically into overarching im-
Materials and Methods plementation “activities” (e.g., understanding the problem, facilitating
Data were extracted from a cluster randomized trial. Therapist/client implementation).
dyads were randomized to AIM-HI or usual care. AIM-HI therapists re- Results
ceived training/consultation for 6 months. Therapists (N=156) reported We identified 40 core competencies which we categorized into 10
K&C at baseline and 6 months. Three K&C subscales were used in ana- implementation activities: Inspiring Stakeholders and Developing Re-
lyses: ASD Knowledge (ASD), (2) Knowledge of EBI ASD strategies lationships; Building Implementation Teams; Understanding the Prob-
(KNOW), and (3) Confidence in EBI ASD strategies (CONF). Repeated- lem; Using Evidence to Inform all Aspects of KT; Assessing the
measures ANOVAs were used in analyses*. Context; Facilitating Implementation; Evaluation; Planning for Sustain-
Results ability; Brokering Knowledge; and Disseminating Evidence. Addition-
A main effect of AIM-HI training was found for all K&C constructs; ally, we identified 5 values or guiding principles for implementation
AIM-HI therapists reported greater increases compared to usual care practice, which emerged from the document review. We are building
an Implementation Practice Core Competency Tool which will be fi- A106

nalized by September. Barriers to implementation of evidence-based treatments for
Conclusions posttraumatic stress disorder at 12-months post training
This presentation will briefly highlight the methods and then focus Mariya Zaturenskaya1, Sebastian Bliss1, Katherine Dondanville2, Brooke
on how to prioritize and select relevant core competencies for pro- Fina1, Vanessa Jacoby1, Jeremy Karp1, Arthur Marsden1
1
jects and individuals. The competencies can be used as a guide to University of Texas Health Science Center at San Antonio, San Antonio,
prioritize capacity building efforts. TX, USA; 2UT Health San Antonio, San Antonio, TX, USA
Correspondence: Mariya Zaturenskaya ([email protected])
A105
Implementation practitioners: what knowledge, skills and abilities Background
do they need? Despite rigorous efforts to disseminate evidence-based treatments
Jenna McWilliam1, Jacquie Brown2 (EBTs) for posttraumatic stress disorder (PTSD) in community settings
1
Triple P International, Brisbane, Australia; 2Families Foundation, and major medical systems such as the Veterans Health Administra-
Hilversum, The Netherlands tion, penetration rates for these treatments have been suboptimal [1-
Correspondence: Jenna McWilliam ([email protected]) 2]. Understanding of provider-related and client-related barriers and
Implementation Science 2020, 15(Suppl 2):A105 challenges to EBT implementation is crucial to improving EBT reach
with community mental health settings. This is a naturalistic study of
Background perceived implementation barriers and challenges for EBTs for PTSD
The increased influence of implementation science has prompted an in a sample of community providers who were trained in cognitive
important question: if an organisation does not have personnel who processing therapy or prolonged exposure for PTSD by the STRONG
are fluent in implementation science how do they apply it? Often an STAR Training Initiative (SSTI), a comprehensive competency-based
implementation expert, intermediary organisation or consultant is en- training program designed to disseminate EBTs.
gaged. But who are they and what knowledge, skills and abilities must Materials and Methods
they have? There is limited literature on the core competencies of im- To date, 42 community-based mental health providers completed a
plementation practitioners including a lack of common terminology survey 12 months after completing an in-person SSTI training work-
and title. However, there is emerging literature on the role, knowledge, shop. The follow-up survey assessed the barriers to initiating
skills and abilities that contribute to effective consultation for imple- evidence-based treatments as well as the challenges with imple-
mentation [1-2]. This presentation draws on this literature, our experi- menting EBTs with PTSD clients.
ence developing the implementation support capabilities of a Results
purveyor/intermediary organisation (Triple P International) [3] learnings At 12-month post-training, 64% of providers endorsed at least one
from Triple P International Implementation Consultants (TPI-ICs), who client-related barrier, while 19% reported at least one therapist-related
for over five years have supported organisations in the application of barrier to initiating an EBT with clients diagnosed with PTSD. The most
implementation science and with implementing organisations and common barrier to initiating treatment was client’s declining the use of
practitioners. The presentation will use data from experience and the an EBT (57% of providers). The barrier endorsed the least was discom-
competencies and coaching literature to promote discussion about re- fort introducing EBTs (0%). When looking at challenges to implement-
quired competencies for implementation practitioners. ing EBT for PTSD, 83% of providers reported at least one challenge,
Materials and Methods with 55% of providers reporting 1-2 challenges, and 29% reporting 3 or
A review was undertaken to examine the role and responsibilities of more. The most common challenges to implementing an EBT included:
the TPI-ICs. This included a review of existing literature; a survey of TPI- client disinterest in engaging in EBT (50%), a difficulty obtaining an ap-
ICs (n=27) and a review of internal support systems and processes. propriate referral (45%), a lack of clients with PTSD on caseload (31%),
Results and a difficulty taking time away from regular work to attend consult-
The following areas were identified as areas of significance for competen- ation call (14%). The challenge endorsed the least was the lack of mo-
cies: Partnering with implementing organisations; determining fit; estab- tivation to use a new treatment (2%).
lishing relationships and roles; facilitating implementation planning; Conclusions
monitoring and evaluating; establishing the innovation as usual practice. The results of this study suggest that the majority of community pro-
Results from the survey identified self-reported levels of confidence and viders report experiencing challenges with both, EBT initiation and im-
competence in knowledge, skills and abilities related to implementation plementation. Understanding barriers to initiation and implementation
consultation. The review of existing systems and process identified areas can guide trainers and organizations in addressing provider concerns
for improvement and increased structure to support desired outcomes. thereby improving dissemination and implementation efforts.
Results were then used to inform the development of TPI-IC Competen-
cies, and a comprehensive IC Management and Support Process. References
Conclusions 1. Marques L, Dixon L, Valentine SE, Borba CPC, Simon NM, Stirman SW.
The presentation will describe knowledge-base, processes and char- Providers’ perspectives of factors influencing implementation of evidence-
acteristics for IC Competencies, IC Management and Support Process. based treatments in a community mental health setting: a qualitative inves-
It aims to promote discussion on future areas of practice develop- tigation of the training—practice gap. Psychol Serv. 2016;(3):322-331
ment as well as exploring ways that additional research could help 2. Rosen CS, Matthieu MM, Stirman SW, Cook JM, Landes S, Bernardy NC, Chard
advance our understanding and further develop the field. KM, Crowley J, Eftekhari A, Finley EP, Hamblen JL, Harik JM, Kehle-Forbes SM,
Meis LA, Osei-Bonsu PE, Rodriguez AL, Ruggiero KJ, Ruzek JI, Smith BN, Trent
References L, Watts BV. A review of studies on the system-wide implementation of
1. Metz A, Louison L, Ward C, Burke K. National Implementation Research Network. evidence-based psychotherapies for posttraumatic stress disorder in the Vet-
Global Implementation Specialist Practice Profile: Skills and Competencies for erans Health Administration. Adm Policy Ment Health. 2016;43(6):957-977.
Implementation Practitioners. 2017. https://www.effectiveservices.org/downloads/
Implementation_specialist_practice_profile.pdf. Accessed 10 August 2017.
2. Proctor EK, Landsverk J, Baumann AA, Mittman BS, Aarons GA, Brownson A107
RC, Glisson C, Chambers D. The implementation research institute: Training future mental health professionals in managing and
training mental health implementation researchers in the United States. adapting practice, an evidence-informed system of care
Implement Sci. 2013;8(1):105. doi:10.1186/1748-5908-8-105. Julia Cox1, Michael Southam-Gerow2
1
3. McWilliam J, Brown J, Sanders MR, Jones L. The Triple P implementation University of California, Los Angeles, Los Angeles, CA, USA; 2Virginia
framework: the role of purveyors in the implementation and Commonwealth University, Richmond, VA, USA
sustainability of evidence-based programs. Prev Sci. 2016;17(5):636-645. Correspondence: Julia Cox ([email protected])
doi:10.1007/s11121-016-0661-4. Implementation Science 2020, 15(Suppl 2):A107
Background adoption, implementation, and sustainability [2-4]. As part of a school-

High quality mental health services do not reach the youth who need based prevention trial, school stakeholders participated in intervention
them, leading to efforts to implement effective treatments more training and implementation with a goal of sustaining the intervention
broadly. One focus of these efforts concerns training the mental health at the schools after the study’s conclusion. This presentation explicates
workforce, of which masters-level social workers represent a large pro- lessons learned via the study of stakeholder involvement, which in-turn
portion. However, the curricula of master’s in social work (MSW) pro- informed subsequent implementation and evaluation efforts.
grams do not often emphasize evidence-based approaches. One Materials and Methods
possible solution is Managing and Adapting Practice (MAP; Practice- Over three years, 20 urban public schools were recruited for a ran-
Wise, LLC), a system that allows clinicians to (1) identify clinically indi- domized trial assessing two wellness programs, one targeting eighth
cated evidence-based programs by searching a growing evidence-base graders’ emotion regulation and decision-making (RAP Club) and the
of randomized controlled trials (RCTs) and (2) build individualized other health education (Healthy Topics). At each school, middle-
evidence-informed treatment plans by focusing on common practice school teachers and school-based mental health providers were re-
elements. MAP may also address the concerns about manual-based cruited as intervention “co-facilitators in training.” One school mental
programs (e.g., inflexibility). Although some MSW programs have inte- health provider per school received training in RAP Club, and one
grated MAP, the benefits of MAP training within MSW education have middle school teacher per school received training in Healthy Topics
not yet been evaluated. This project evaluated multiple mechanisms of (N = 40 across all schools). Stakeholder attendance and engagement
training [1] in a semester-long MSW-focused MAP course relative to during intervention training, supervision calls, and intervention ses-
curriculum-as-usual control at a large public university. sions were recorded. School characteristics (e.g., organizational
Materials and Methods health), stakeholder interviews, and process notes further informed
Participants were advanced MSW students (mean age = 27, SD = 5.8; our investigation of stakeholder engagement in training and
92.3% women; 59% white) either enrolled in the MAP course (n = 17) implementation.
or enrolled in curriculum-as-usual (n = 22). The MAP course was co- Results
taught by an expert MAP trainer and a MAP-trained social worker. Most stakeholders attended intervention training; however, attend-
Pre- and post-semester, participants completed a battery that in- ance on supervision calls was limited. Two-thirds of school personnel
cluded: (1) role-plays with standardized patients that were video- were regularly present during intervention sessions, but fewer than
taped and coded using the Therapy Observational Coding System of half actively participated. Stakeholder participation and engagement
Child Psychotherapy – Revised Strategies scale [2]; (2) a written task increased each year of the trial. Individual- and school-level factors
that was subsequently coded to assess participants’ clinical decision- were related to participation in training, supervision, and implemen-
making skills during different phases of a standardized case; and (3) tation. Findings will be used to refine personnel training and supervi-
attitudinal factors that may be predictive of future MAP usage, such sion (e.g., augmenting structure, explicating goals) for the final year
as attitudes toward evidence-based practice [3] and the acceptability of trial implementation.
and feasibility of MAP [4]. Conclusion
Results Evaluating stakeholder involvement in intervention training and im-
Results indicate significant uptake of cognitive and behavioral therapeutic plementation has been critical in informing this research team’s ap-
strategies in the MAP condition. Overall, participants endorsed positive proach to stakeholder training and supervision to support
attitudes toward evidence-based practice broadly and MAP specifically. implementation and sustainability. Our findings illustrate both chal-
Conclusions lenges and opportunities for increased stakeholder involvement in
Findings may be used to inform the development of more effective implementation. Factors influencing stakeholder participation,
evidence-informed curriculum for masters-level clinical programs and assessed during controlled trials, can be leveraged to inform subse-
future workforce training initiatives. Methodological considerations quent evaluations, as well as program adoption, implementation,
may inform advances in instrumentation to measure multidimen- and sustainability.
sional training outcomes. Trial registration: Clinicaltrials.gov NCT03906682
1. McLeod BD, Cox JR, Jensen-Doss A, Herschell A, Ehrenreich-May J, Wood 1. Lyon AR. Implementation Science and Practice in the Education Sector.
JJ. Proposing a mechanistic model of training and consultation. Clin Psy- 2017. https://education.uw.edu/sites/default/files/Implementation Science
chol Sci Pract. 2018;25. doi:10.1111/cpsp.12260 Issue Brief 072617.pdf. Accessed 20 March 2019.
2. McLeod BD, Smith MM, Southam-Gerow MA, Weisz JR, Kendall PC. Measur- 2. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-
ing treatment differentiation for implementation research: the Therapy implementation hybrid designs. Med Care. 2012;50:217–26.
Process Observational Coding System for Child Psychotherapy Revised Strat- 3. Goldstein H, Olswang L. Is there a science to facilitate implementation of
egies scale. Psychol Assess. 2015;27(1):314-25. doi:10.1037/pas0000037 evidence-based practices and programs? Evid Based Commun Assess
3. Aarons GA. Mental health provider attitudes toward adoption of Interv. 2017;11:55–60.
evidence-based practice: the Evidence-Based Practice Attitude Scale 4. Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ,
(EBPAS). Ment Health Serv Res, 2004;6(2):61-74. Aarons GA, Hoagwood KE, Evans AC, Hurford MO, Rubin R, Hadley T,
4. Chafouleas SM, Briesch AM, Neugebauer SR, Riley-Tillman TC. Usage rating Mandell DS, Barg FK. A multi-level examination of stakeholder perspectives
profile – intervention (revised). Storrs, CT: University of Connecticut; 2011. of implementation of evidence-based practices in a large urban publicly-
funded mental health system. Adm Policy Ment Health. 2016;43:893–908.
A108
Lessons learned: a data-driven approach to supervision and A109
training of stakeholders Who takes advantage of training initiatives: are we just preaching
Stephanie Moore, Laura Clary, Kimberly Arnold, Steven Sheridan, Tamar to the choir and whistling in the wind?
Mendelson Brigid Marriott, Jack Andrews, Kristin Hawley
Johns Hopkins University, Baltimore, MD, USA University of Missouri, Columbia, MO, USA
Correspondence: Stephanie Moore ([email protected]) Correspondence: Brigid Marriott ([email protected])
Few evidence-based practices (EBPs) are successfully installed into Numerous implementation initiatives have endeavored to bridge the
school settings [1]. Integrating implementation considerations into early research-to-practice gap [1-2]. However, the reach of these implemen-
stages of intervention evaluation and collaborating with relevant stake- tation initiatives has rarely been studied. In the current study, we de-
holders are recommended to reduce the gap between EBP evaluation, scribe a county-wide youth mental health (MH) initiative supported by
a voter-approved sales tax. This initiative aims to improve access to ef- effects on patient health alongside implementation strategy effects
fective youth MH services by providing free training, consultation, and on implementation outcomes [2-4], though the role of context as a
support in evidence-based practices (EBPs) to MH service providers. third independent variable (IV) is incompletely specified.
The current study has three aims: 1) describe the providers reached by Materials and Methods
the initiative, 2) examine which training activities providers engage in Our objective is to expand the hybrid effectiveness-implementation
(i.e., formal workshops; learning collaboratives; individual consultation), framework to include mixtures of all three types of IVs: intervention,
and 3) explore differences in providers (e.g., discipline; attitudes; know- implementation strategy, and context. We propose to use I to repre-
ledge) who do and do not invest in training activities. sent the IV of intervention, IS to represent implementation strategy,
Materials and Methods and C to represent context.
Participants (N = 523) were community MH providers who completed Results
a web-based baseline assessment prior to registering for the EBP The expanded framework specifies nine two-variable hybrid designs:
trainings. Measures included demographics, clinical practice informa- I/is, I/IS, IS/i, IS/c, IS/C, C/is, C/i, I/C, and I/c. We describe four in detail:
tion, self-reported confidence, organizational climate [3], and EBP I/is, IS/c, IS/C, and C/is. We also specify seven three-variable hybrid
knowledge [4], attitudes [5], and practice [6]. designs that follow from the two-variable designs. We argue that
Results many studies already meet our definition of two- or three-variable
The initiative reached over 500 providers who were part of over 100 hybrids.
different organizations and private practices. Registered providers Conclusions
were predominantly master’s level (N = 277, 53.06%), representing Our proposal builds naturally from the typology proposed by Curran
social work (N=178, 34.10%), counseling (N=140, 26.82%), psychology et al. [2], but offers a more complete and clear specification of de-
(N=61, 11.69%), and other MH disciplines (N=143, 27.34%). Some 159 signs that might be of interest to implementation researchers. We
(34.40%) were fully licensed MH providers, 69 (13.19%) post-degree need studies that are designed and powered to measure the
but unlicensed, 119 (22.75%) student trainees, and 176 (33.65%) implementation-related effects of variations in contextual determi-
none or other types of licensure (e.g., RN, MD). Providers on average nants, both to advance the science and to optimize delivery of inter-
had been providing MH services for 8.13 years (SD = 8.52, range = 0 ventions in the real world. Prototypical implementation studies that
to 45). Registered providers participated most frequently in formal evaluate the effectiveness of an implementation strategy, in isolation
workshops (69.79%, N=365,) and less often in small group learning from its context, risk perpetuating the persistent gap between evi-
collaboratives (7.07%, N=37) and individual consultation (6.88%, N= dence and practice, as they will not generate essential context-
36). Initial findings showed significant, positive associations between specific knowledge around implementation, scale-up, and de-
baseline EBP attitudes (r=.11, p=.01) and knowledge (r=.13, p<.01) implementation.
and the number of formal workshops attended.
The initiative reached a high proportion of MH providers and organi- 1. Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A,
zations; however, far fewer actually participated in any training activ- Hofmann B, Wahlster P, Polus S, Burns J, Brereton L. Making sense of
ities. The more in-depth, personal training and support components complexity in context and implementation: The Context and
were the least utilized. Implications for voluntary implementation ini- Implementation of Complex Interventions (CICI) framework. Implement
tiatives within community MH care will be discussed. Sci. 2017;12(1):21.
2. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C: Effectiveness-
References implementation hybrid designs: Combining elements of clinical effective-
1. Dorsey S, Berliner L, Lyon AR, Pullmann MD, Murray LK. A statewide ness and implementation research to enhance public health impact.
common elements initiative for children’s mental health. J Behav Health Med Care. 2012; 50(3):217.
Serv Res. 2016;43(2):246–261. 3. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A,
2. Jensen-Doss A, Hawley KM, Lopez M, Osterberg LD. Using evidence- Griffey R, Hensley M. Outcomes for implementation research: conceptual
based treatments: The experiences of youth providers working under a distinctions, measurement challenges, and research agenda. Adm Policy
mandate. Prof Psychol Res Pract. 2009;40(4):417. Ment Health. 2011;38(2):65-76.
3. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational 4. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
context for EBP implementation: The development and validity testing of MM, Proctor EK, Kirchner JE. A refined compilation of implementation
the Implementation Climate Scale (ICS). Implement Sci. 2014;9(1):157. strategies: Results from the Expert Recommendations for Implementing
4. Stumpf RE, Higa-McMillan CK, Chorpita BF. Implementation of evidence- Change (ERIC) project. Implement Sci. 2015;10(1):21.
based services for youth: Assessing provider knowledge. Behav Modif.
2009;33(1):48–65.
5. Aarons GA, Cafri G, Lugo L, Sawitzky A. Expanding the domains of A111
attitudes towards evidence-based practice: The evidence based practice The use of the PARIHS framework in implementation research and
attitude scale-50. Adm Policy Ment Health. 2012;39(5):331–340. practice – a citation analysis of the literature
6. Cho E, Wood PK, Taylor EK, Hausman EM, Andrews JH, Hawley KM. Anna Bergström1, Anna Ehrenberg2, Ann Catrine Eldh3, Ian
Evidence-based treatment strategies in youth mental health services: Re- Graham4, Kazuko Gustafsson1, Gillian Harvey5, Alison Kitson6, Jo Rycroft-
sults from a national survey of providers. Adm Policy Ment Health. Malone7, Lars Wallin2
1
2019;46(1):71–81. Uppsala University, Uppsala, Sweden; 2Dalarna University, Falun,
Sweden; 3Linköping University, Linköping, Sweden; 4University of
Ottawa, Ottawa, Ontario, Canada; 5University of Adelaide, Adelaide,
A110 Australia; 6Flinders University, Adelaide, Australia; 7Bangor University,
Expanding hybrid designs for implementation research: Bangor, Wales, United Kingdom
intervention, implementation strategy, and context Correspondence: Anna Bergström ([email protected])
Christopher Kemp1, Bradley Wagenaar1, Emily Haroz2 Implementation Science 2020, 15(Suppl 2):A111
1
University of Washington, Seattle, WA, USA; 2Johns Hopkins University,
Baltimore, MD, USA Background
Correspondence: Christopher Kemp ([email protected]) The Promoting Action on Research Implementation in Health Ser-
Implementation Science 2020, 15(Suppl 2):A110 vices (PARIHS) framework was developed two decades ago and con-
ceptualizes successful implementation (SI) as a function (f) of the
Background evidence (E) nature and type, context (C) quality and the facilitation
Successful implementation reflects the interplay between interven- (F), [SI = f (E,C,F)] [1-4]. Despite a growing number of citations of the-
tion, implementation strategy, and context [1]. Hybrid effectiveness- oretical frameworks including the PARIHS, details of how theoretical
implementation studies allow investigators to assess intervention frameworks are used remains largely unknown. This review aimed to
enhance the understanding of the breadth and depth of the use of Materials and Methods
the PARIHS framework. We are reviewing published academic journal articles to identify the
Materials and Methods state of quantitative measurement of health policy implementation.
This citation analysis departed from four core articles representing To guide the systematic measures review, we combined two frame-
the key stages of the framework’s development. The citation search works impacting policy implementation: 1) for internal context, the
was performed in Web of Science and Scopus. After exclusion, we Consolidated Framework for Implementation Research; [3] and 2) for
undertook an initial assessment aimed to identify articles using PARI external context, Bullock’s policy implementation determinants
HS and not only referencing any of the core articles. To assess this, framework (under review).
all articles were read in full. Further data extraction included captur- We are applying Lewis et al.’s measures review protocol and PAPERS
ing information about where (country/countries and setting/s) PARI rating system [4]. We searched these databases: CINAHL Plus, Med-
HS had been used, as well as categorizing how the framework was line, PsychInfo, PAIS, ERIC, and Worldwide Political. The four search
applied. Also, strengths and weaknesses, as well as efforts to validate strings included multiple search terms for: health, public policy, im-
the framework, were explored in detail. plementation, and measurement. We will code measures to imple-
Results mentation outcomes [5] and predictors. Inclusion criteria: 1) empirical
The citation search yielded 1,163 articles. After applying exclusion cri- study of the implementation of public policies already passed or ap-
teria, 1,059 articles were read in full, and the initial assessment proved addressing physical or behavioral health; 2) quantitative self-
yielded a total of 259 articles reported to have used the PARIHS report or archival measures utilized; 3) peer-reviewed journal publica-
framework. These articles were included for data extraction. The tion 1995 through April 2019; and 4) English language text.
framework had been used in a variety of settings and in both high-, Results
middle- and low-income countries. With regards to types of use, 28% We will screen abstracts April-June 2019. In July-August 2019 we will re-
used the PARIHS in planning and delivering an intervention, 49% in view and extract full texts. We will present yields, characteristics of in-
data analysis, 55% in the evaluation of study findings, and/or 46% in cluded articles, and description of several identified quantitative policy
any other way. Further analysis showed that its actual application implementation outcome measures. We will show plans for fall/winter
was frequently partial, and generally not well elaborated. 2019 pragmatic measure rating, summarization, and web-based posting.
Conclusions We seek feedback from policy implementers and researchers on
In line with previous citation analysis of the use of theoretical frame- remaining procedures. We especially want feedback on design of a pub-
works in implementation science we found a rather superficial de- licly available web-based summary of identified measures and pragmatic
scription also of the use of the PARIHS. Thus, we propose the properties to ensure usefulness to policy implementers and researchers.
development and adoption of reporting guidelines on how frame- We are collaborating with SIRC on methodology and dissemination.
work(s) are used in implementation studies, with the expectation Conclusions
that it enhances the maturity of implementation science. The measures summary is intended to stimulate further assessment
of health policy implementation outcomes and predictors to help
References practitioners and researchers spread evidence-informed policies to
1. Kitson A, Harvey G, McCormack B. Enabling the implementation of improve population health.
evidence-based practice: a conceptual framework. Qual Health Care.
1998;7(3):149-58. References
2. Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, 1. The Community Guide. Guide to community preventive services. 2019.
Estabrooks C. Ingredients for change: revisiting a conceptual framework. https://www. thecommunityguide.org. Accessed 25 March 2019.
BMJ Quality Saf. 2002;11(2):174-80. 2. Watson DP, Adams EL, Shue S, Coates H, McGuire A, Chesher J, Jackson J,
3. Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A. Omenka OI. Defining the external implementation context: an integrative
An exploration of the factors that influence the implementation of evi- systematic literature review. BMC Health Serv Res. 2018;18(1):209
dence into practice. J Clin Nurs. 2004;13(8):913-24. 3. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC.
4. Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Fostering implementation of health services research findings into
Evaluating the successful implementation of evidence into practice using practice: a consolidated framework for advancing implementation
the PARiHS framework: theoretical and practical challenges. Implement science. Implement Sci. 2009;4:50.
Sci. 2008;3:1. 4. Lewis CC, Mettert KD, Dorsey CN, Martinez RG, Weiner BJ, Nolen E,
Stanick C, Halko H, Powell BJ. An updated protocol for a systematic
review of implementation-related measures. Syst Rev. 2018;7(1):66.
A112 5. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A,
How are health policy implementation outcomes measured Griffey R, Hensley M. Outcomes for implementation research: conceptual
quantitatively? a review protocol distinctions, measurement challenges, and research agenda. Adm Policy
Peg Allen1, Cole Hooley1, Meagan R. Pilar1, Cara C. Lewis2, Kayne D. Ment Health. 2011; 38(2):65-76.
Mettert2, Caitlin N. Dorsey2, Jonathan Purtle3, Stephanie
Mazzucca1, Alexandra B. Morshed1, Ana Baumann1, Maura M. Kepper1,
Ross C. Brownson1 A113
1
Brown School, Washington University in St. Louis, St. Louis, MO, USA; Development and evaluation of an instrument to measure fidelity
2
Kaiser Permanente Washington Health Research Institute, Seattle, WA, to implementation of collaborative care in primary care clinics
USA; 3Drexel University, Philadelphia, PA, USA Erin LePoire, Anna Ratzliff, Diane Powers, Deborah J. Bowen
Correspondence: Peg Allen ([email protected]) University of Washington, Seattle, WA, USA
Implementation Science 2020, 15(Suppl 2):A112 Correspondence: Erin LePoire ([email protected])
Background
Evidence about effective strategies in clinical care and population Background
health is growing, with a number of evidence-based policy ap- Collaborative Care (CoCM) has been shown to be an effective way to
proaches now recommended in the Community Guide [1] and treat depression and other mental illness in primary care and other
other systematic reviews. But understanding lags on how best to settings. Evaluating whether or not clinics that receive training and
implement recommended policies to reap the full population technical assistance to implement CoCM maintain fidelity to the core
health benefits. Information is limited on how to quantitatively components proven in other research to predict better patient out-
measure policy implementation outcomes [2]. To address this gap comes has received little attention. A scalable measure clinics can
a systematic review has begun to identify and rate quantitative use to measure fidelity is needed. This analysis discusses the creation
measures of health policy implementation outcomes and and evaluation of an instrument specifically designed to measure
predictors. multiple domains of CoCM fidelity.
Materials and Methods useful (27.6%) along with having patient specific care plans (24.6%)
Development of the CoCM fidelity tool occurred in three steps. Step and using electronic reminders (18.8%).
one was development of a rubric that was utilized during in-person Pharmacists at facilities which did not implement one or more strat-
site visits with a group of HRSA-funded clinics. In Step two the rubric egies reported barriers like time-consuming/not enough staff
domains were adapted based on the outcomes of these site visits (43.6%), they didn’t believe this would work (22.7%), to implement
and incorporated into a qualitative interview guide administered to they needed help from other services/departments (18.6%) and some
care managers trained in CoCM within the preceding six months to pharmacists believed this was inappropriate work for a pharmacist
assess fidelity. Step three focused on converting the rubric to a self- (14.9%).
administered format so it can be used by organizations to self-assess Conclusions
fidelity to core CoCM components. Pharmacists perceived provider education as the most useful strategy
Results to monitor medication safety for their patients. Formative evaluation
In Step one, clinics had an average rubric score of 3.09 “core features focuses on the identification and adoption of best practices to im-
Implemented” (range 2.16-4.59) out of a possible 5.0 which indicates prove medication safety for the Veterans.
“exceptional Implementation.” Step two transcript data from qualita-
tive interviews revealed that care managers are able to link their
clinic’s current CoCM processes to core CoCM concepts in the fidelity A115
rubric. Findings from development of the rubric and qualitative inter- Defining and developing measures to assess public health
views will be presented. The Step three self-administered rubric will program sustainability
be tested in 24 clinics across the United States participating in a Sarah Moreland-Russell, Rebecca Vitale, Elizabeth Zofkie
CoCM implementation. We will compare this data to clinic outcomes Brown School, Washington University in St. Louis, St. Louis, MO, USA
in order to determine validity. Correspondence: Sarah Moreland-Russell ([email protected])
Conclusions Implementation Science 2020, 15(Suppl 2):A115
A self-administered instrument to assess CoCM fidelity in primary
care clinics is feasible and further evaluation will allow us to connect Background
use of the rubric to patient-level clinical outcomes and provider-level Many recent Dissemination and Implementation Science studies have
outcomes among all members of the CoCM team (care manager, pri- neglected to observe what happens to programs once they have
mary care provider, psychiatric consultant). been implemented. This has contributed to the lack of a cohesive
and succinct definition of sustainability for public health programs.
While certain studies define sustainability as the continuation of pro-
A114 grammatic activities over time, others conceptualize sustainability as
Measurement of implementation strategies for pharmacy benefits the continued delivery of benefits to target populations and the
management MUET initiatives to optimize medication maintenance of collaborative structures within communities [1]. Ul-
management timately, conflicts between these definitions disrupt the continuity of
Anju Sahay1, Francesca Cunningham2, Peter Glassman2, Von Moore2, Muriel program sustainability research and focus. With public health funding
Burk2, Parisa Gholami1, Shoutzu Lin1, Brian Mittman3, Paul Heidenreich1,4 in perpetual jeopardy, a cohesive, solidified definition of program
1
MedSafe QUERI Program, Palo Alto VA Health Care System, Palo Alto, sustainability has never been more necessary.
CA, USA; 2VA Office of Pharmacy Benefits Management Services, Materials and Methods
Washington, DC, USA; 3Kaiser Permanente, CA, USA; 4Stanford University, The study began with an extensive systematic literature review of pro-
Stanford, CA, USA gram sustainability research, including empirical research, case studies,
Correspondence: Anju Sahay ([email protected]) fieldwork, and commentaries. This process outlined various proposed
Implementation Science 2020, 15(Suppl 2):A114 definitions of sustainability and cataloged organizational metrics tied to
sustainability outcomes. The target audience for the current study uti-
Background lized evidence-based state tobacco control (TC) programs. Therefore,
Effective implementation strategies are critical for understanding and the second part of the methodology included consultations and inter-
improving outcomes. VA’s Office of Pharmacy Benefits Management views with key tobacco control specialists, sustainability experts, aca-
Service (PBM) has a national Medication Use Evaluation Tracker demics, and practice-oriented professionals. These interviews were
(MUET) web application designed to provide close to real-time then cross-referenced with the literature to find commonalities be-
summary-level patient data (monthly to quarterly) to reduce poten- tween theory and practice. Finally, the study team collected and ana-
tially unsafe or unnecessary medication. Focusing on the develop- lyzed federal progress reports submitted annually by these TC
mental phase of formative evaluation, in the context of five current programs. The items outlined in these reports were referenced back to
MUET initiatives, we collaborated with PBM to understand their use, the previously identified sustainability metrics [2-3]. The organizational
and the value of seven pre-identified strategies to implement them: metrics described in the literature were aligned to the evidence-based
provider education, academic detailing, electronic reminders, patient Program Sustainability Framework. This framework defines the internal
specific care plan, draft orders, patient mailings and calling patients. and external factors operationalized into eight domains that affect a
These five MUET initiatives were Dimethyl Fumarate (DMF), New Min- program’s capacity for sustainability [4-5].
eralocorticoid Receptor Antagonist (MRA), Prasugrel or Ticagrelor Results
Treatment Duration >12 months (PRAT), Women of Childbearing Age Through this process, institutionalization emerged as the primary measure
on Warfarin (WoW) and Direct Oral Anticoagulants (DOAC). of sustainability found in both the literature and dialogue. To this effect,
Materials and Methods the establishment of a program through formal organizational rules and
In collaboration with PBM, in 2017 for the DMF initiative (n=143) and in funding was widely perceived to ensure the delivery of continued pro-
2018 for the remaining four initiatives (n=142) all VISN Pharmacy Exec- gram initiatives and benefits. The results further distinguish between pro-
utives (VPEs) emailed web-based surveys to a designated pharmacist at grammatic, organizational, community-level factors, and funder support,
each of their facilities. Goal was to understand which among the seven suggesting sustainability planning must account for institutional scope.
strategies were being used by the facilities to implement the specific Conclusions
MUET initiative, and their perceived value. We also assessed barriers for This concise definition will enable valid, empirical comparisons of
facilities not using the implementation strategies. Response rates were sustainability across programs in various public health contexts.
as follows: 2017 (n=127, 89.0%) and 2018 (n=131, 93.2%).
Results References
The most commonly used implementation strategies by pharmacists 1. Scheirer MA. Is sustainability possible? A review and commentary on
were provider education (26.6%), patient mailings (23.5%), using empirical studies of program sustainability. Am J Eval. 2005; 26:320-47.
electronic reminders (21.2%) and entering draft orders (14.3%). Com- 2. Vitale R, Blaine T, Zofkie E, Moreland-Russell S, Combs T, Brownson R,
paratively, pharmacists perceived provider education as being most Luke D. Developing an evidence-based program sustainability training
curriculum: a group randomized, multi-phase approach. Implement Sci. 3. Zatzick DF, Russo J, Darnell D, Chambers DA, Palinkas L, Van Eaton E,
2018;13(1):126. Wang J, Ingraham LM, Guiney R, Heagerty P, Comstock B. An
3. Savaya R, Spiro S. Predictors of sustainability of social programs. Am J effectiveness-implementation hybrid trial study protocol targeting post-
Eval. 2012; 33(1):26-43 traumatic stress disorder and comorbidity. Implement Sci. 2016;11(1):58.
4. Luke D, Calhoun A, Robichaux C, Elliott M, Moreland-Russell S. The Pro- 4. Moyers TB, Martin T, Manuel JK, Miller WR, Ernst D. Revised global scales:
gram Sustainability Assessment Tool: A new instrument for public health motivational interviewing treatment integrity 3.1. 1 (MITI 3.1. 1). Unpublished
programs. Prev Chronic Dis. 2014;11:130-184. manuscript, University of New Mexico, Albuquerque, NM. 2010.
5. Schell S, Luke D, Schooley M, Elliott M, Mueller N, Bunger A. Public health 5. Beidas RS, Cross W, Dorsey S. Show me, don’t tell me: Behavioral
program capacity for sustainability: a new framework. Implement Sci. rehearsal as a training and analogue fidelity tool. Cogn Behav Pract.
2013;8(15). 2014;21(1):1-1.
A116 A117
Pragmatic measurement of the quality of healthcare provider Getting to fidelity: identifying core components of implementation
patient-centered behavior change counseling using the behavior facilitation strategies
change counseling index (BECCI) Correspondence: Jeffrey Smith ([email protected])
Doyanne Darnell, Kaylie Diteman, Dylan Fisher, Lea Parker, Allison Behavioral Health QUERI, North Little Rock, AR, USA
Engstrom, Christopher Dunn Implementation Science 2020, 15(Suppl 2):A117
University of Washington, Seattle, WA, USA
Correspondence: Doyanne Darnell ([email protected]) Background
Implementation Science 2020, 15(Suppl 2):A116 To ensure appropriate transfer of successful implementation strat-
egies from research to policy and practice, it is important to use tools
Background or processes to measure and support fidelity to a given strategy’s
Training healthcare providers in Motivational Interviewing or similar core components [1]. Unfortunately, this aspect of implementation
patient-centered behavioral interventions is increasingly popular [1]; science is underdeveloped and infrequently applied [2]. Implementa-
however, gold-standard methods to assess skill acquisition and on- tion facilitation (IF) is a dynamic strategy involving interactive
going quality assessment are laborious and impractical in busy problem-solving and support to help clinical personnel implement
healthcare settings. We examined the utility of a brief measure re- and sustain a new program or practice that occurs in the context of
quiring modest training, the Behavior Change Counseling Index a recognized need for improvement and a supportive interpersonal
(BECCI) [2], to pragmatically capture provider skill in patient-centered relationship [3]. Identifying core components of IF is a foundational
alcohol counseling. step in efforts to develop tools to assess fidelity to the strategy.
The present study includes a multidisciplinary sample of routine First, we conducted a scoping literature review to identify the range of
trauma center providers (N = 69) trained to counsel trauma patients activities applied in IF strategies. PubMed, CINAHL, and Thompson Sci-
about risky alcohol use as part of a 25-site National Institutes of entific Web of Science databases were searched for English-language
Health-funded pragmatic trial of a collaborative care intervention [3]. articles that included the term “facilitation” or other commonly used
Providers were predominantly White (79%) females (87%) with at terms for the strategy published from January 1996 – December 2015.
minimum a bachelor’s degree. Providers completed a pre-training Initially, 1,489 citations/abstracts were identified and screened for rele-
20-minute standardized patient role-play in which they counseled a vance by two independent reviewers. Ultimately, 135 articles (from 94
patient actor about alcohol use. At the end of the role-play, the stan- studies) were identified for abstraction of data on facilitator characteris-
dardized patient actor completed the brief (<5 minute) 12-item tics and roles/activities, clinical setting, patient population, clinical
BECCI measure. Audio recordings of the role-plays were subsequently innovation targeted for implementation, and implementation out-
coded by an objective rater using the Motivational Interviewing comes. Next, we engaged an Expert Panel in a rigorous 3-stage modi-
Treatment Integrity Scale (MITI), a longer gold-standard measure that fied Delphi process to develop consensus on core IF activities for high
requires intensive training [4]. No previous studies have directly com- complexity and low complexity clinical innovations in three implemen-
pared the BECCI and the MITI. We examined correlation coefficients tation phases (pre-implementation, implementation, sustainment).
(Spearman’s rho for skewed MITI variables) between overall BECCI Results
scores and MITI empathy and summary scores. Based on review of the literature for the 94 studies, 32 distinct IF ac-
Results tivities were identified. The Expert Panel identified 8 of the 32 IF ac-
The overall BECCI scores were highly and statistically significantly (p < tivities as core for the Pre-Implementation Phase, 8 core IF activities
.05) correlated with key patient-centered counseling style MITI scores for the Implementation Phase, and 4 core IF activities for the Sustain-
(empathy r = .70, spirit r = .74, MI-adherent r = .51) and the behavioral ment Phase. A prototype IF Fidelity Tool based on the core activities
count scores of percent open questions. (rs = .56) and reflection-to- has been developed for piloting.
question ratio (rs = .60). BECCI scores were moderately and statistically Conclusions
significantly correlated with percent open questions (rs = .33). Core IF activities were identified based on a comprehensive literature
Conclusions review and a rigorous consensus development process with an ex-
The BECCI is a pragmatic measure of patient-centered behavior change pert panel. Effective transfer of successful IF strategies from research
counseling that may be useful for routine use in healthcare settings to to policy and practice requires tools to help ensure fidelity to core
assess counseling quality. Given that the BECCI does not require exten- components of the strategy.
sive training it may be used by either a trainer/supervisor or peer to
pragmatically assess various training sessions (e.g., behavioral rehearsal References
[5]) as well real patient interactions (live or audio-recorded). 1. Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting
complex behaviour change interventions: the need for a scientific
References method. Implement Sci 2009; 4:40. doi: 10.1186/1748-5908-4-40
1. Lundahl B, Moleni T, Burke BL, Butters R, Tollefson D, Butler C, Rollnick S. 2. Slaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of
Motivational interviewing in medical care settings: a systematic review documentation and reporting of fidelity to implementation strategies: a
and meta-analysis of randomized controlled trials. Patient Educ Couns. scoping review. Implement Sci 2015;10:129. doi: 10.1186/s13012-015-0320-3
2013;93(2):157-68. 3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
2. Lane C, Huws-Thomas M, Hood K, Rollnick S, Edwards K, Robling M. MM, Proctor EK, Kirchner JE. A refined compilation of implementation
Measuring adaptations of motivational interviewing: the development strategies: results from the Expert Recommendations for Implementing
and validation of the behavior change counseling index (BECCI). Patient Change (ERIC) project. Implement Sci. 2015;10(1):21. doi: 10.1186/s13012-
Educ Couns. 2005;56(2): 166-73. 015-0209-1
A118 5. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the

Implementation fidelity and sustainability of midwife-led antenatal framework method for the analysis of qualitative data in multi-
consultation: preliminary results disciplinary health research. BMC Med Res Methodol. 2013;13(1):117.
Anja Siegle1, Friederike zu Sayn-Wittgenstein2, Martina Roes3
1
University of Witten Herdecke, Witten, North Rhine-Westphalia, Germany;
2
University of Applied Science Osnabrück, Osnabrück, Germany; 3German A119
Centre for Neurodegenerative Diseases, Bonn, Germany Preventing facilitator burnout: strategies for more sustainable
Correspondence: Anja Siegle ([email protected]) process improvement
Implementation Science 2020, 15(Suppl 2):A118 Tanya Olmos-Ochoa1, David Ganz1,2, Jenny Barnard1,
Lauren Penney3,4, Neetu Chawla1
1
Background Veterans Affairs Greater Los Angeles, Los Angeles, CA, USA; 2University
All over the word medical interventions in child-birth are increas- of California Los Angeles, Los Angeles, CA, USA; 3South Texas Veterans
ing [1]. Since 2014, in Germany, there exists a national nursing Health Care System, San Antonio, TX, USA; 4University of Texas Health
expert standard to promote physiological child-birth, which de- Science Center, Houston, TX, USA
mands antenatal consultation conducted by midwives who are Correspondence: Tanya Olmos-Ochoa ([email protected])
employed by a hospital. National expert standards define an Implementation Science 2020, 15(Suppl 2):A119
evidence-based and practitioners consented quality level using
Donabedian’s model of structure, process and outcome criteria Background
[2]. During the pilot implementation period (6 month in 2015) in A substantial evidence base supports the use of practice facilitation
13 German hospitals, the antenatal consultation was not evalu- as an effective strategy to enable implementation of evidence-based
ated. Thus, to what extent antenatal consultation was imple- practices and related quality improvement (QI) efforts in learning
mented and how implementation success looks like remained healthcare systems [1-3]. Yet, challenges with implementing and
unclear. The aim of this study is to investigate implementation fi- maintaining facilitation exist and may impede efforts to grow and
delity (adherence, participant responsiveness) [3] and sustainabil- sustain an experienced facilitator workforce. This study identifies po-
ity (benefits, institutionalization, development) [4] of antenatal tential challenges facilitators may experience when working with QI
consultation in two hospitals. teams in real world settings and recommends strategies to address
Materials and Methods these challenges.
A mixed-methods design has been chosen, including a quantitative Materials and Methods
content analysis of consultation documents (n=154) and 34 qualita- The Coordination Toolkit and Coaching (CTAC) project is a VA-funded
tive semi structured interviews with midwifes, pregnant women, phy- QI initiative to improve patient experience of care coordination in
sicians and managers in two hospitals in Germany. A descriptive primary care. Using a cluster-randomized design, 12 primary care
analysis was undertaken for the documents. The interviews were ana- clinics were randomized to either a passive strategy (access to the
lyzed using framework analysis [5]. CTAC online toolkit) or an active strategy (distance-based coaching
Results plus access to the toolkit). Over a 12-month period, two facilitators
Adherence is higher in hospital B, which had a longer timeframe delivered weekly, one-hour coaching calls to six clinics implementing
for implementation than hospital A. Furthermore, hospital B had a QI project of the clinic’s choice. Data sources included facilitator re-
already experience in consultations. Participant responsiveness flections catalogued after all coaching calls (n=232) and notes from
was very positive in both hospitals. In both hospitals, the inter- debrief sessions between facilitators.
viewed persons saw benefits. Institutionalization is also given in Results
both hospitals, but differs regarding time frame and consultation We identified nine facilitation stressors: lack of progress/follow-
process. A need for evaluation of the change over time of the through; changes to the coached team; emotion/frustration directed
needs of women and tailoring interventions to these needs was at the facilitator; mismatched expectations between the facilitator
seen in hospital B, but not in hospital A. and coached team; managing project timeline and deliverables; sup-
Conclusions porting QI methods and data collection; managing team dynamics;
Implementing antenatal consultation in German hospitals is feasible promoting effective communication; and documenting implementa-
but it needs more time than the pilot implementation of 6 month. tion and facilitation processes. Given these stressors, we recommend
Based on the study, it seems that a longer time period (~ 12 months) that facilitators: continually re-assess process improvement activities
and a positive attitude to adapt to new developments increases im- and QI methods (e.g., aligning goals with project timeline); moderate
plementation outcomes. Furthermore, flexibility in applying the 4- discussions to help anticipate and resolve common challenges to
step implementation model, securing resources, and convincing all process improvement (e.g., staffing turnover, within-team conflict);
stakeholders might have had an impact on feasibility. Additionally, support teams with appropriate data collection and analysis; and set
there is a need to evaluate antenatal consultation after the woman aside time to self-reflect (e.g., debrief sessions), discuss (e.g., with a
gave birth. co-facilitator), and make necessary adjustments to their facilitation
process.
Conclusions
References Understanding how facilitation affects facilitators and providing facili-
1. Miller S, Abalos E, Chamillard M, Ciapponi A, Colaci D, Comandé D, tators with tools to address stressors are essential for sustainability of
Diaz V, Geller S, Hanson C, Langer A, Manuelli V, Millar K, Morhason- QI and other process improvement efforts, and for continued use of
Bello I, Castro CP, Pileggi VN, Robinson N, Skaer M, Souza JP, Vogel facilitation as an implementation strategy. Identifying facilitation
JP, Althabe F. Beyond too little, too late and too much, too soon: a stressors and strategies to overcome them may enhance the devel-
pathway towards evidence-based, respectful maternity care world- opment and maintenance of an experienced facilitator workforce to
wide. Lancet. 2016;388(10056):2176-2192. support the next generation of process improvement.
2. DNQP. Methodisches Vorgehen zur Entwicklung, Einführung und
Aktualisierung von Expertenstandards in der Pflege und zur Entwicklung References
von Indikatoren zur Pflegequalität auf Basis von 1. Harvey G, Kitson A. Implementing evidence-based practice in healthcare:
Expertenstandards. Osnabrück 2015. a facilitation guide. New York: Routledge; 2015.
3. Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on 2. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated
fidelity of implementation: implications for drug abuse prevention in framework for the successful implementation of knowledge into practice.
school settings. Health Educ Res. 2003;18(2):237-256. Implement Sci. 2016;11:33.
4. Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. The sustainability 3. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of
of healthcare innovations: a concept analysis. J Adv Nurs. practice facilitation within primary care settings. Ann Fam Med.
2015;71(7):1484-1498. 2012;10(1):63-74.
A120 5. Eaton J, McCay L, Semrau M, Chatterjee S, Baingana F, Araya R, Ntulo C,

What makes an enabling context for mental health delivery? Thornicroft G, Saxena S. Scale up of services for mental health in low-
differential workload adjustment to sustain task-sharing delivery income and middle-income countries. Lancet. 2011;378(9802):1592-1603.
across education and health sectors in a low resource setting doi:10.1016/S0140-6736(11)60891-X
Grace Woodard1, Noah Triplett1, Christine Gray2, Rosemary Meza1, Prerna
Martin1, Leah Lucid1, Kathryn Whetten2, Gabrielle Jamora1, Augustine A121
Wasonga3, Cyrilla Amanya3, Shannon Dorsey1 The benefits of ad hoc adaptations in implementation science:
1
University of Washington, Seattle, WA, USA; 2Duke University, Durham, community-based practices can support delivery of a family
NC, USA; 3 ACE Africa Kenya, Bungoma, Kenya therapy intervention in Eldoret, Kenya
Correspondence: Grace Woodard ([email protected]) Bonnie Kaiser1, Julia Kaufman2, Johnathan Taylor Wall3, Elsa Friis-
Implementation Science 2020, 15(Suppl 2):A120 Healy2, Byron Powell4, David Ayuku5, Eve Puffer2
1
University of California San Diego, San Diego, CA, USA; 2Duke
Background University, Durham, NC, USA; 3Duke Global Health Institute, Durham, NC,
Evidence suggests mental health interventions can be effectively de- USA; 4Brown School, Washington University in St. Louis, St. Louis, MO,
livered via task-sharing in low-resource settings with high need for USA; 5Moi University, Eldoret, Kenya
mental health interventions; [1–3] however, research is needed to Correspondence: Bonnie Kaiser ([email protected])
identify approaches to sustain the delivery in these settings. [4] Implementation Science 2020, 15(Suppl 2):A121
We examine qualitative reports of lay counselors experienced in deliver- Background
ing group-based trauma-focused cognitive behavioral therapy (TF-CBT) A key question in implementation science is how to balance adapta-
for orphaned children and adolescents in western Kenya. We analyze im- tion and fidelity in translating interventions to new settings. Most
plementation policies and practices (IPPs) associated with delivering TF- psychological interventions carried out in low-and-middle-income
CBT in the health and education sectors in order to determine impactful countries (LMICs) were originally developed in high-income coun-
and feasible IPPs to sustain task-sharing delivery in a low-resource setting. tries. There is growing consensus regarding the importance of, and
Eighteen teachers and 18 community health volunteers (CHVs; N = 36) processes for, planned adaptations so that interventions are deliv-
participated in qualitative interviews after delivering two groups of TF- ered in contextually sensitive ways. However, little research has ex-
CBT. Thematic coding for IPPs was conducted by a team including one amined ad hoc adaptations, or those that occur spontaneously in the
PI. Interviews were double-coded and discussed to consensus; a third course of intervention delivery. A key question is whether ad hoc ad-
coder was consulted when discordant. Less than half (n = 17) of the inter- aptations ultimately contribute to or detract from intervention effect-
views were in Swahili and were coded by a member of the study team iveness. This study aimed to (a) identify ad hoc adaptations made
fluent in Swahili and English; then, all Swahili interviews were translated during delivery of a family therapy intervention and (b) assess
verbally and discussed to consensus with a PI. whether they promoted or hindered intervention goals.
Results Materials and Methods
Workload adjustment emerged as a critical and feasible IPP for sustaining Tuko Pamoja (Swahili: “We are Together”) is an evidence-based family
task-sharing in the education sector: 83% of teachers (n = 15/18) indi- therapy intervention aiming to improve family dynamics and mental
cated that limited or no workload adjustment was a barrier to implemen- health, being delivered in Eldoret, Kenya. Tuko Pamoja is delivered
tation. However, it was minimally important in the health sector, with by lay counselors, who are afforded a degree of flexibility in the way
only 17% of CHVs (n = 3/18) indicating workload adjustment was a bar- they present intervention content and the practices they use in ther-
rier. Teachers at urban schools (n = 6) were more likely to report work- apy sessions. This study used transcripts of therapy sessions with 14
load adjustment as a facilitator than teachers at rural schools (n = 12). families to develop a taxonomy of ad hoc adaptations used by coun-
Examples of workload adjustments include adjustments of individual selors. We first identified and characterized these adaptations. Then,
schedules (68% urban teachers versus 0% rural teachers), adjustment of we evaluated to what extent they were in the spirit of the interven-
school schedules (50% versus 8%), and exemption from meetings (50% tion or went against the goals of the intervention.
versus 17%). Sustainable implementation strategies are needed to ad- Results
dress large-scale health inequities in low-resource settings. [5] We found Ad hoc adaptations included the incorporation of metaphors and
differential use and importance of workload adjustment in two sectors proverbs, religious content, self-disclosure, examples and role
(both unique and overlapping), which enables tailored implementation models, discussing interpersonal relationships outside of the family,
support depending on the sector and setting (urban, rural). and community dynamics and resources. For the most part, practices
Conclusions were Tuko Pamoja-promoting, though Tuko Pamoja-contrary prac-
Our results can inform future implementation and sustainment of tices were also identified.
task-sharing interventions in low resource settings. Conclusions
Trial Registration ClinicalTrials.gov NCT01822366 Identifying helpful ad hoc adaptations and incorporating them into
interventions could improve acceptability, feasibility, and
References effectiveness.
1. Murray LK, Skavenski S, Kane JC, Mayeya J, Dorsey S, Cohen JA, Trial Registration ClinicalTrials.gov NCT03360201
Michalopoulos LT, Imasiku M, Bolton PA. Effectiveness of trauma-focused
cognitive behavioral therapy among trauma-affected children in Lusaka, A122
Zambia: a randomized clinical trial. JAMA Pediatr. 2015;169(8):761-769. Adaptations to an evidence-based health promotion practice
doi:10.1001/ jamapediatrics.2015.0580 implemented nationally in routine mental health settings
2. World Health Organization. Task Shifting. Global Recommendations and Kelly Aschbrenner1, Gary Bond2, Sarah Pratt1, Stephen Bartels3
1
Guidelines. Geneva, Switzerland; 2008. https://www.who.int/ Geisel School of Medicine at Dartmouth, Hanover, NH, USA; 2Weststat,
workforcealliance/knowledge/resources/ taskshifting_guidelines/en/. Rockville, MD, USA; 3Massachussetts General Hospital, Boston, MA, USA
3. O’Donnell K, Dorsey S, Gong W, Ostermann J, Whetten R, Cohen JA, Correspondence: Kelly Aschbrenner (kelly.aschbrenner@dartmouth.
Itemba D, Manongi R, Whetten K. Treating maladaptive grief and edu)
posttraumatic stress symptoms in orphaned children in Tanzania: group- Implementation Science 2020, 15(Suppl 2):A122
based trauma-focused cognitive–behavioral therapy. J Trauma Stress.
2014;27(6):664-671. doi:10.1002/jts.21970 Background
4. Munodawafa M, Mall S, Lund C, Schneider M. Process evaluations of task There is increasing recognition that local program adaptations may be
sharing interventions for perinatal depression in low and middle income instrumental to sustaining evidence-based interventions in routine clin-
countries (LMIC ): a systematic review and qualitative meta-synthesis. ical practice. [1] However, few empirical studies have documented nat-
2018:18(1):205. urally occurring adaptations made during the implementation process
by providers and agencies in health care settings. Our research directly address PCMH implementation challenges included improving patient-
addresses the SIRC conference theme, “Where the Rubber Meets the provider communication. This paper examines EBQI-PCMH effectiveness
Road,” by identifying and categorizing provider-initiated adaptations to for improving patient-provider communication over time, compared with
an evidence-based health promotion practice implemented nationally standard PCMH implementation (PCMH-only).
in routine mental health care settings. Materials and Methods
Materials and Methods We used a non-randomized stepped wedge design in which sites en-
Our team conducted semi-structured telephone interviews with pro- tered in three phases, 6-8 quarters apart. We compared Veterans’ expe-
gram staff from 35 behavioral health organizations 24 months after riences at 10 VHA practices (seven EBQI-PCMH versus three PCMH-only)
they implemented InShape, a manualized evidence-based health pro- on patient-provider communication. In PCMH-only transformation, pro-
motion practice for persons with serious mental illness, within the viders and staff in all primary care sites received training in motivational
context of an NIMH-funded implementation study. The interview interviewing and patient-centered communication. In EBQI-PCMH sites
protocol included questions that assessed core fidelity components researchers partnered with clinical leaders to support local develop-
of the InSHAPE model, with probes used to explore any adaptations ment of QI projects addressing patient-provider communication. We
made to core components. An adaptation was defined as a change used repeated cross-sections of nationally-administered patient experi-
to the intervention content or method of delivery that was not speci- ence surveys from 2009-2015 to assess EBQI-PCMH impacts (N=34,193).
fied in the original treatment manual. We explored the reasons why Outcome measures included patient ratings of four provider communi-
an adaptation was made as well as who initiated it at the agency. cation skills: 1) explaining information (EXPLAIN), 2) listening (LISTEN),
Two investigators independently reviewed interview transcripts to 3) showing respect (RESPECT), and 4) spending enough time (TIME),
identify adaptations to the program. and rated as optimal for scores 9-10 vs lower. Predictors included time,
Adaptations to InSHAPE included hybrid individual and group pro- EBQI-PCMH implementation, and length of exposure to EBQI-PCMH.
grams, home-based exercise programs, technology-based enhance- We compared EBQI-PCMH to PCMH-only practice sites using multi-
ments, such as mobile fitness apps and wearable activity trackers, and level, multivariate modelling controlling for patient and site characteris-
use of peer support specialists to deliver program components. The tics and weighted for non-response.
next level of analysis will involve classifying adaptations as fidelity- Results
consistent (i.e., changes that do not significantly alter core model ele- Patient ratings of all provider communication skills improved with
ments) and fidelity-inconsistent adaptations (i.e., changes that reduce longer exposure to EBQI-PCMH, adjusting for patient and site charac-
the delivery of core model elements), [2] and categorizing the drivers teristics. Each additional quarter of exposure to EBQI-PCMH was asso-
of adaptations (e.g., client-driven, financially-driven). We will then evalu- ciated with improved odds of optimal communication: 2.75%
ate the impact of these adaptations on client-level health outcomes. increase in EXPLAIN, 2.95% increase in LISTEN, 2.70% increase in
Results RESPECT, and 2.29% increase in TIME. For example, over the span of
Evidence-based practices are often modified by agencies when trans- the evaluation period (23 quarters), the predicted probability of
lated from research environments to real world health care settings. higher rating for EXPLAIN was 76%, 73%, and 68%, for EBQI-PCMH
[3] However, the impact of these adaptations on client health out- for Phase 1, 2, 3 versus 63% for PCMH-only.
comes is not well understood. Conclusions
Conclusions EBQI-PCMH that engages leaders, providers and staff in researcher-
By identifying and characterizing site-specific adaptations, we will be supported QI to accelerate patient centered transformation can be
able to explore the relationship of adaptations to program-and effective in improving patient-provider communication.
participant-level outcomes and sustainability of the InSHAPE program
at the completion of the study. References
1. Stewart M, Brown J, Donner A, et al. The impact of patient-centered care
References on outcomes. J Fam Prac. 2000;49(9):796-804.
1. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability 2. Rubenstein LV, Stockdale SE, Sapir N, Altman L, Dresselhaus T, Salem-
framework: addressing the paradox of sustainment amid ongoing Schatz S, Vivell S, Ovretveit J, Hamilton AB, Yano EM. A patient-centered
change. Implement Sci. 2013;8:117. primary care practice approach using evidence-based quality improve-
2. Wiltsey Stirman S, A Gutner C, Crits-Christoph P, Edmunds J, Evans AC, ment: rationale, methods, and early assessment of implementation. J Gen
Beidas RS. Relationships between clinician-level attributes and fidelity- Intern Med. 2014; 29(2):589-597.
consistent and fidelity-inconsistent modifications to an evidence-based 3. Reddy A, Canamucio A, Werner RM. Impact of the patient-centered med-
psychotherapy. Implement Sci. 2015;10(1):115. ical home on veterans’ experience of care. Am J Manag Care.
3. Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, 2015;21(6):413-421.
Hartman M, Fernandez ME, Mullen PD. A systematic review of adapta-
tions of evidence-based public health interventions globally. Implement
Sci. 2018;13(1):125. A124
Facilitators’ perspectives on facilitation successes and challenges
in a quality improvement initiative
A123 Neetu Chawla1, David Ganz1, Jenny Barnard1, Lauren Penny2, Tanya
Evidence-based quality improvement for accelerating patient- Olmos-Ochoa1
1
centered medical home implementation: impact on patient- VA Greater Los Angeles, Los Angeles, CA, USA; 2Veteran’s Health
provider communication Administration South Texas, San Antonio, TX, USA
Alexis Huynh, Danielle Rose, Martin Lee, Catherine Chanfreau- Correspondence: Neetu Chawla ([email protected])
Coffinier, Karleen Giannitrapani, Lisa Rubenstein, Susan Stockdale Implementation Science 2020, 15(Suppl 2):A124
Veterans Health Administration, Bedford, MA, USA
Correspondence: Alexis Huynh ([email protected]) Background
Implementation Science 2020, 15(Suppl 2):A123 Quality improvement efforts and implementation science use facilita-
tion as an effective implementation strategy. Limited work has exam-
Background ined the successes and challenges of this strategy from the
High-quality patient-provider communication is foundational to patient- facilitator’s perspective, which could shed light on evaluation of facili-
centered care and a core component of the Patient-Centered Medical tation effectiveness and success of implementation efforts [1-2].
Home (PCMH) [1]. The PCMH model could disrupt patient-provider com- Materials and Methods
munication by shifting communication responsibilities to non-provider We conducted thematic analysis on qualitative data from the Coord-
PCMH team members [2,3]. We introduced Evidence-Based Quality Im- ination Toolkit and Coaching (CTAC) project, a multi-site quality im-
provement for PCMH transformation (EBQI-PCMH) at seven Veterans Af- provement initiative within the VA healthcare system. Two CTAC
fairs (VHA) primary care practices. Quality improvement (QI) methods to facilitators (“coaches”) logged their perceptions of the successes and
challenges in a “reflection” template completed after each weekly Materials and Methods
one-hour coaching call over the 12-month project period. Given The Quality Implementation Framework [3] and evidence-based sys-
CTAC is ongoing, this analysis examines the successes and challenges tem for innovation support [4] were used to inform the development
identified for one coached site (n=41 reflections). Two members of of implementation supports at the agency and long-term care home
the project team independently coded the reflections to identify level. Five agency staff conducted readiness conversations and deliv-
common themes related to successes and challenges resulting from ered group-based online implementation training sessions to leads
the coaching process. from 44 long-term care homes. Two online surveys were adminis-
Results tered to the leads from each home to assess fidelity to the program
We identified 15 total themes related to successes or challenges, of recommendations and to gather feedback on the quality of the train-
which six were categorized as both a success and a challenge: Pro- ing sessions.
ject participation and engagement; Communication between coach Results
and coached team members; Managing team dynamics; Conflict Participation rates were variable, with only 29% of long-term care
resolution; Time management; and Call productivity (e.g., progress homes attending all three scheduled sessions. Of the homes that
on CTAC deliverables, project tasks, or products). For example, for continued with the program, over 70% had adopted implementation
the theme of moderating team dynamics, coaches described obtain- strategies designed to support readiness and buy-in for the practice
ing “buy-in from all the stakeholders and facilitating the discussions changes. In April, data from the final survey will be analyzed to de-
between them” as a success but simultaneously noted that “balan- scribe what program strategies were used by participating homes
cing nursing priorities/frustrations with administrative staff’s prior- based on a fidelity measurement tool established for the program.
ities/frustrations” was a challenge. Similarly, for the theme of Conclusions
communication, coaches noted “encouraging more people to speak An online and group-based implementation support model has
up” as a success but also “getting them to be more verbal during the proven to be efficient in reaching more homes; however, there is a
call” as a challenge. need to assess the implications of this higher touch approach on
Conclusions program fidelity. This model has raised questions about how an
Facilitation is increasingly used as an implementation strategy, yet intermediary can support stakeholders secure buy-in, plan for sus-
the successes and challenges experienced by facilitators during the tainability, and be inspired to adopt approaches from implementa-
facilitation process are not well-defined [3]. Given that effective facili- tion science.
tation may lead to positive implementation outcomes, facilitation
success should be examined carefully. Our findings indicate that References
some aspects of coaching can be assessed as both successes and 1. Chambers A, MacFarlane S, Zvonar R, Evans G, Moore JE, Langford BJ,
challenges, highlighting the complexity of the facilitation process. Augustin A, Cooper S, Quirk J, McCreight L, Garber G. Recipe for
Better understanding facilitation effectiveness will support a more antimicrobial stewardship success: Using intervention mapping to
nuanced conceptualization of how implementation efforts that use develop a program to reduce antibiotic overuse in long-term care. Infect
facilitation either fail or succeed. Control Hosp Epidemiol. 2019;40(1):24-31.
Trial Registration ClinicalTrials.gov NCT03063294 2. Brown AK, Chambers A, MacFarlane S, Langford B, Leung V, Quirk J,
Schwartz KL, Garber G. Reducing unnecessary urine culturing and
References antibiotic overprescribing in long-term care: outcomes of an implemen-
1. Harvey G, McCormack B, Kitson A, Lynch E, Titchen A. Designing and tation science informed before and after study. CMAJ Open.
implementing two facilitation interventions within the ‘Facilitating 2019;7(1);E174–E181.
Implementation of Research Evidence (FIRE)’ study: a qualitative analysis 3. Meyers DC, Durlak JA, Wandersman A. The quality implementation
from an external facilitators’ perspective. Implement Sci. 2018;13(1):141. framework: a synthesis of critical steps in the implementation process.
2. Rycroft-Malone J, Seers K, Eldh AC, et. al. A realist process evaluation Am J Community Psychol. 2012;50(3-4):462-80.
within the Facilitating Implementation of Research Evidence (FIRE) cluster 4. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for
randomised controlled international trial: an exemplar. Implement Sci. innovation support for implementing innovations with quality: tools,
2018;13(1):138. training, technical assistance, and quality assurance/quality improvement.
3. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of Am J Community Psychol. 2012;50(3-4):445-459.
practice facilitation within primary care settings. Ann Fam Med.
2012;10(1):63-74.
A126
Early lessons from formative evaluation of an implementation
A125 intervention to improve reach of evidence-based psychotherapies
Building an impactful implementation support model to scale-up a for PSTD
quality improvement program in long-term care Princess Ackland1,2, Shannon Kehle-Forbes1,2,3, Matthew Yoder3,4, Robert
Andrea Chaplin, Sam MacFarlane Orazem1, Nancy Bernardy3,5, Jessica Hamblen3,5, Craig Rosen3,6, Siamak
Public Health Ontario, Toronto, Ontario, Canada Noorbaloochi2, Barbara Clothier1, Sean Nugent1, Paula Schnurr3,5, Nina
Correspondence: Andrea Chaplin ([email protected]) Sayer1,2
Implementation Science 2020, 15(Suppl 2):A125 1
Minneapolis VA Health Care System, Minneapolis, MN, USA; 2University
of Minnesota, Minneapolis, MN, USA; 3National Center for PTSD, VA Palo
Background Alto Health Care System, Palo Alto, CA, USA; 4Medical University of
A provincial agency is working to scale an organizational improve- South Carolina, Charleston, SC, USA; 5Dartmouth College, Hanover, NH,
ment program that supports long-term care homes overcome bar- USA; 6Stanford University, Stanford, CA, USA
riers to aligning with evidence based practices related to the Correspondence: Princess Ackland ([email protected])
assessment and management of urinary tract infections [1-2]. These Implementation Science 2020, 15(Suppl 2):A126
practices, if addressed, could help reduce the overuse of antibiotics
that are contributing to antibiotic resistance and increased risk of Background
antibiotic side effects. The initial pilot of this program involved We used toolkit-guided external facilitation to improve access
agency staff delivering in-person support to an implementation team to evidence-based psychotherapies (EBPs) for PTSD in two out-
that was established in 12 long-term care homes. With over 600 patient PTSD clinics with low reach of EBPs (≤15% Veterans
long-term care homes in the province of Ontario, a more efficient im- with PTSD) at baseline. The Promoting Action on Research Im-
plementation support model was needed. The purpose of this phase plementation in Health Services framework informed the imple-
of the project was to apply best practices from implementation sci- mentation strategy and evaluation [1]. The objective of this
ence to develop and evaluate a new implementation model that study is to describe preliminary results from the formative
could be used to scale-up the program. evaluation.
Materials and Methods Interviews were analyzed using a directed content analysis approach
Developmental evaluation data included pre-site visit interviews with [4]. We assessed reliability and validity of our coding frame through
4-6 key informants and baseline data on the primary implementation joint coding by four analysts on two transcripts. Data were then
outcome—EBP reach, defined as the percentage of unique patients mapped to the levels of the social connection framework. Each
who receive a session of Prolonged Exposure or Cognitive Processing coded transcript was discussed in depth until consensus on coding
Therapy in the PTSD clinic. Implementation-focused evaluation data definitions was reached. Following this process, six analysts inde-
was extracted from a facilitation log used to track facilitation activ- pendently coded between 30-40 transcripts. Data were checked and
ities, time spent in these activities and contact with the champion entered into the NVivo software package for ease of organization
and other local staff. Progress-focused evaluation data included monthly and reporting.
audit and feedback reports on EBP reach, based on administrative data, Results
and narrative review of goal attainment recorded in the site-specific Social disconnection, described as “loneliness” and “social isolation”
guide. Interpretive evaluation data included post-intervention interviews by stakeholders, led patients to the ED for problems not always re-
with the same key informants interviewed at baseline. Interviews were lated to their physical health. These definitions mapped to the struc-
analyzed using rapid turn-around approach [2]. tural level of the social connection framework. Innovations involving
Results home visit programs, elder services interventions, workflow changes
EBP reach more than doubled during the 6-month intervention in the ED, and regular telephone follow-ups provided functional level
period in both clinics. Clinic A achieved its reach goal of 20% in emotional and tangible support. Stakeholders did not mention rela-
month 2 and continued to increase linearly to 36% at month 6. Clinic tionship distress or quality of relationships in describing social dis-
B’s reach increased slightly then plateaued until month 6 when it connection or hospital innovations.
achieved its goal of 25%. External facilitation hours were 70% greater Conclusions
in Clinic A. Clinic A implemented organizational changes consistently Innovations to address high ED use, according to stakeholders, pro-
over the 6 months while Clinic B enacted significant changes shortly vided functional level emotional and tangible support to address
before the 6-month reach increase. Clinic A’s champion was more structural level definitions of social disconnection. Future work
committed and empowered to make organizational changes com- should examine the sustainability of these innovations in a value-
pared with Clinic B’s champion. Implementation strategies associated based healthcare climate, and the effectiveness of these programs
with reach at both sites included audit and feedback reports, an in- on reducing ED utilization.
person site visit at project launch, and toolkit resources.
Improvement trajectories may not be consistent across sites. Imple- 1. The 191st General Court of the Commonwealth of Massachusetts. Acts of
mentation interventions should vary in duration according to local 2012. Chapter 224. An act improving the quality of health care and
champion characteristics. Toolkit-guided external facilitation accom- reducing costs through increased transparency, efficiency and
panied by a strong local champion has the potential to help clinics innovation. https://malegislature.gov/Laws/SessionLaws/Acts/2012/
reorganize to improve reach of EBPs to patients. Chapter224. Accessed 31 March 2019.
2. Louis C, Bachman S, Roby D, Melby L, Rosenbloom D. The transformation
References of community hospitals through the transition to value-based care: les-
1. Rycroft-Malone J. Promoting action on research implementation in health sons from Massachusetts. Am J Account Care. 2017;5(4):26-30.
services (PARIHS). In: Rycroft-Malone J, Bucknall T, editors. Models and 3. Holt-Lundstadt J, Robles TF, Sbarra DA. Advancing social connection as a
frameworks for implementing evidence-based practice: linking evidence public health priority in the United States. Am Psychol. 2017;72(6):517-
to action. Hoboken, NJ: John Wiley & Sons; 2010. p. 109-135. 530.
2. Hamilton AB. Qualitative methods in rapid turn-around health services re- 4. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis.
search. VA HSR&D Cyberseminar, December 2013. Qual Health Res. 2005; 15(9):1277-1288.
A127 A128
Addressing social disconnection among frequent users of Evaluation of The Implementation Game©: a learning and planning
community hospital emergency departments: a statewide resource
implementation evaluation Correspondence: Melanie Barwick ([email protected])
Rani Elwy1, Elisa Koppelman1, Victoria Parker2, Chris Louis1 The Hospital for Sick Children, Toronto, Ontario, Canada
1
Boston University, Boston, MA, USA; 2University of New Hampshire, Implementation Science 2020, 15(Suppl 2):A128
Durham, NH, USA
Correspondence: Rani Elwy ([email protected]) Background
Implementation Science 2020, 15(Suppl 2):A127 The presentation will share evaluation findings for a new planning
and learning resource to support implementation of evidence into
Background care. Implementation is a complex process with many moving parts,
Chapter 224 of the Commonwealth of Massachusetts Acts of 2012 [1] and many practitioners and organizations struggle to do it success-
authorized Massachusetts to establish the Community Hospital Accel- fully. The Implementation Game© (TIG) supports autonomous, self-
eration, Revitalization, and Transformation (CHART) investment pro- direct implementation by simplifying the process into five main com-
gram. The Massachusetts Health Policy Commission (HPC) oversees ponents to provide an implementation planning experience for an
the CHART program, which awarded $120 million to 27 community identified scenario or implementation endeavor. It is based on key
hospitals to develop innovations aimed at enhancing the delivery of implementation theories, models, and frameworks [1-4].The Imple-
efficient, effective care, and readying them for value-based care [2]. mentation Game is relevant to any discipline because the concepts
Objective: Through a contract with the HPC, we conducted an imple- are high level. The Game components include a game board, playing
mentation evaluation of CHART innovations between 2016-2018. cards, and an implementation worksheet to capture the plan. The
Through this evaluation, we examined how CHART stakeholders de- goal is either to learn, or to plan, or both. The presentation will pro-
scribed social disconnection, a public health priority, and which vide an overview of the Game and preliminary evaluation data.
levels of a social connection framework CHART innovations ad- Materials and Methods
dressed (structural, functional, quality or multilevel) [3] among fre- An online survey has been shared with 36 (and counting) individuals
quent emergency department (ED) users. who either purchased or received a copy of The Implementation
Materials and Methods Game© beginning in December 2018. The survey captures evidence
Qualitative interviews with 236 stakeholders (hospital managers, of use, usefulness, spread, quality, and satisfaction.
CHART providers, staff, and community partners) one year post CHAR Results
T implementation were audiorecorded and transcribed verbatim. Currently in data collection.
Conclusions life skills in high schools [3], we will discuss how consultation can
Evaluation results will be used to refine the Game and to inform a support implementation to meet community, teacher and student
prototype for an online software platform that is under development needs in the context of policy changes, as well as potential short-
by the author. comings of consultation.
Trial Registration ClinicalTrials.gov NCT00336180
References
1. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC. References
Fostering implementation of health services research findings into 1. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
practice: a consolidated framework for advancing implementation MM, Proctor EK, Kirchner JE. A refined compilation of implementation
science. Implement Sci. 2009;4:50. strategies: results from the Expert Recommendations for Implementing
2. Myers DC, Durlak JA, Wandersman A. The Quality Implementation Change (ERIC) project. Implement Sci. 2015;10(1):21. doi:10.1186/s13012-
Framework: a synthesis of critical steps in the implementation process. 015-0209-1.
Am J Community Psychol. 2012;50(3-4):462-80. 2. Caldwell L, Smith E, Collins L, Graham J, Lai M, Wegner L, Vergnani T,
3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu Matthews C, Jacobs J. Translational research in South Africa: Evaluating
MM, Proctor EK, Kirchner JE. A refined compilation of implementation implementation quality using a factorial design. Child Youth Care Forum.
strategies: results from the Expert Recommendations for Implementing 2012;41(2):119-136.
Change (ERIC) project. Implement Sci. 2015;10(1):21. 3. Maharajh LR, Nkosi T, Mkhize MC. Teachers’ experiences of the
4. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, implementation of the Curriculum and Assessment Policy Statement
Griffey R, Hensley M. Outcomes for implementation research: conceptual (CAPS) in three primary schools in KwaZulu Natal. Africa’s Public Serv
distinctions, measurement challenges, and research agenda. Adm Policy Deliv Perform Rev. 2017;4(3): 371. doi:10.4102/apsdpr.v4i3.120.
Ment Health. 2011;38(2):65-76. doi: 10.1007/s10488-010-0319-7.
A130
A129 Adoption of trauma-focused interventions within rural schools
Understanding when consultation supports teachers in Heather Halko1, Kaoru Powell1, Erika Burgess1, Cameo Stanick2, Kaitlyn
implementing a prevention program in South African high schools: Ahlers1, Anisa Goforth1
1
moderators and outcomes University of Montana, Missoula, MT, USA; 2Hathaway-Sycamores Child
Mojdeh Motamedi1, Linda Caldwell2,3, Edward Smith2,3, Lisa Wegner3, and Family Services, Los Angeles, CA, USA
Joachim Jacobs3 Correspondence: Heather Halko ([email protected])
1
University of California, San Diego, CA, USA; 2The Pennsylvania State Implementation Science 2020, 15(Suppl 2):A130
University, University Park, PA, USA; 3University of the Western Cape,
Cape Town, South Africa Background
Correspondence: Mojdeh Motamedi ([email protected]) High rates of childhood trauma exposure (over 68%) create signifi-
Implementation Science 2020, 15(Suppl 2):A129 cant concern given the negative outcomes associated with trauma-
related symptoms [1-2]. Numerous trauma-focused evidence-based
Background practices (EBPs) have been developed; however, little is known about
Despite implementation research on supporting evidence-based preven- why school systems, especially those serving rural areas, adopt (or do
tion programs in high-income countries, research is lacking on consult- not adopt) trauma-focused EBPs. This qualitative study explored fac-
ation support in schools in low-resourced countries like high schools tors that might influence the adoption of trauma-focused interven-
surrounding Cape Town, South Africa [1]. This study is part of a larger fac- tions among clinicians working in rural schools using two
torial design implementation trial of HealthWise, a teacher taught pro- implementation science frameworks: the Consolidated Framework for
gram for preventing youths’ risky sexual and substance use behaviors [2]. Implementation Research (CFIR) and the Implementation Outcome
Materials and Methods Framework (IOF) [3-4].
After initial randomization, 22 schools with 33 teachers received the Materials and Methods
consultation condition while 26 schools with 41 teachers did not. The A semi-structured protocol was used to interview school-based clini-
consultation condition included three meetings between the consult- cians (N = 12) about their knowledge, views, and adoption of
ant and a teacher representative per school, text message reminders, trauma-focused interventions. Specific attention was given to IOF
support kits with prepared HealthWise materials, and lesson plans inte- outcomes known to influence innovation adoption (i.e., acceptability,
grating HealthWise content. Teachers self-reported how much content appropriateness, and feasibility) [5]. Transcripts were double coded
they delivered and adapted, and students’ interests in HealthWise les- using a deductive content analysis approach and a CFIR- and IOF-
sons during 9th grade. Observer coded videos captured teachers’ fidel- based coding manual.
ity to HealthWise curriculum. School risk was calculated using publicly Results
available data on school and community safety, poverty, density and Every participant (100%) reported adopting some form of mental
geographical location. Post-intervention qualitative interviews with the health intervention to treat symptoms of posttraumatic stress within
consultant, 11 teachers and 4 principals expand on quantitative find- their school setting, though only 25% had adopted a trauma-focused
ings and broader policy and community priorities. EBP. One participant (8.33%) was also working in a school that de-
Results clined an opportunity to adopt the practice of delivering trauma-
Based on as-treated regression analyses, teachers in the consultation focused care. Thematic analyses revealed that most participants re-
condition reported delivering more HealthWise content (B = .13, p < ported the same acceptability and appropriateness factors as both fa-
.01) but did not differ in their observed fidelity. Moderation analyses cilitators and barriers to adoption of trauma-focused interventions in
found teachers with lower educational degrees who received the rural schools. Nine participants (75%) believed that it was not feasible
consultation condition reported more student interest in HealthWise to implement trauma-focused EBPs within their current school sys-
(B = -.17, p < .01), and teachers in higher risk schools that received tem. Several CFIR constructs (e.g., cosmopolitanism, structural charac-
the consultation condition reported more adaptation (B = .22, p < teristics, leadership engagement, access to knowledge and
.01). Initial qualitative findings suggest there was a need to adapt, es- information, available resources, relative priority, self-efficacy) were
pecially to address higher risk school needs. Additionally, the consul- commonly identified as influencing the feasibility of implementing
tant’s interview suggested racial, educational, and gender differences trauma-focused interventions within a rural school.
may play a role in teachers’ receptivity to consultation. Conclusions
Conclusions The acceptability and appropriateness of delivering trauma-focused
Findings suggest even a low dose of consultation support can facili- care within school settings appears to positively influence the adop-
tate implementation outcomes in this context. As this study occurred tion of trauma-focused interventions within rural schools. However,
during South African education policy changes regarding teaching limited feasibility of implementing trauma-focused EBPs within rural
schools might be negatively influencing adoption. These results have Conclusions

the capacity to inform a targeted approach to select implementation This paper illuminates important findings with implications for imple-
strategies that could enhance the adoption of trauma-focused EBPs mentation research and practice. First, negative misalignment was
within schools, thereby increasing the accessibility of trauma-focused found for over half of the schools, indicating a likely need to improve
care in rural areas. communication and collaboration across levels. Second, although
fewer in number, positive alignment and positive misalignment was
References noted, indicating opportunities to study the factors associated with
1. Copeland WE, Keeler G, Angold A, Costello EJ. Traumatic events optimal alignment on key organizational implementation constructs.
and posttraumatic stress in childhood. Arc Gen Psychiatry. 2007;64:577-
584. References
2. Loeb J, Stettler EM, Gavila T, Stein A, Chinitz S. The Child 1. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across
Behavior Checklist PTSD scale: Screening for PTSD in young children with systems and organizations to develop a strategic climate for evidence-
high exposure to trauma. J Trauma Stress. 2011;24(4):430-434. based practice implementation. Annu Rev Public Health. 2014; 35(1):255-
3. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC. 274. doi:10.1146/annrev-publhealth-032013-182447
Fostering implementation of health services research findings into 2. Aarons GA, Ehrhart MG, Torres EM, Finn NK, Beidas R. The humble leader:
practice: a consolidated framework for advancing implementation Association of discrepancies in leader and follower ratings of
science. Implement Sci. 2009;4:50. implementation leadership with organizational climate in mental health.
4. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Psychiatr Serv Wash DC. 2017;68(2):115-122. doi:10.1176/
Griffey R, Hensley M. Outcomes for implementation research: conceptual appi.ps.201600062
distinctions, measurement challenges, and research agenda. Adm Policy 3. Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, Aarons GA.
Ment Health. 2011;38(2):65-76. Assessing organizational implementation context in the education sector:
5. Chor KH, Wisdom JP, Olin SC, Hoagwood KE, Horwitz SM. Predictors of confirmatory factor analysis of measures of implementation leadership,
innovation adoption. Adm Policy Ment Health. 2015;42(5):545-573. climate, and citizenship. Implement Sci. 2018;12(1):5.
A131 A132
The impact of stakeholder alignment of the organizational LIFT Together with Boys Town: an implementation system
implementation context in schools bridging schools, providers, and researchers
Elissa Picozzi1, Chayna Davis1, Jill Locke1, Mark Ehrhart2, Eric Brown3, Clay Jasney Cogua1, W. Alex Mason2
Cook4, Aaron Lyon1 1
Boys Town - LIFT Together, Boys Town, NE, USA; 2Boys Town Child and
1
University of Washington, Seattle, WA, USA; 2University of Central Family Translational Research Center, Boys Town, NE, USA
Florida, Orlando, FL, USA; 3University of Miami, Coral Gables, FL, USA; Correspondence: Jasney Cogua ([email protected])
4
University of Minnesota, Minneapolis, MN, USA Implementation Science 2020, 15(Suppl 2):A132
Correspondence: Elissa Picozzi ([email protected])
For over 100 years, Boys Town has provided services for children
Background who have suffered adverse experiences, trauma, and other chal-
Organizational factors are critical to successful implementation of lenges [1]. Since 2012, Boys Town has been working to implement a
evidence-based behavioral health interventions. Previous research has comprehensive prevention strategy that goes beyond serving indi-
demonstrated that the alignment or misalignment between leadership vidual youth and families to impacting targeted populations to build
and providers on organizational constructs (e.g., implementation lead- well-being through the LIFT Together program [2].
ership) impacts successful implementation [1]. In particular, positive Materials and Methods
misalignment, which occurs when leaders rate the implementation LIFT Together is a community-based implementation system that
context significantly lower than the staff, may facilitate the successful convenes schools, service providers, and researchers to facilitate the
implementation of evidence-based practices, and negative misalign- delivery of a multi-tier, multi-component intervention package. The
ment may hinder these processes. This paper examines alignment be- intervention package uses school [3] - and family-based [4-6] pro-
tween school administrators and their staff on implementation grams to generate school-wide impact. Outcomes are measured at
leadership (i.e., specific leadership behaviors that support or inhibit ef- the school population level (such as preventing and reducing school
fective implementation) and implementation climate (i.e., shared norms disciplinary referrals or increasing parental engagement at schools),
and expectations among staff related to implementation). instead of upon individual children and families. This intervention
Materials and Methods system is being implemented within highly vulnerable communities
The OASIS study collected data for alignment in 35 schools (6 school in three sites: South Omaha, Nebraska; North Las Vegas, Nevada; and
districts in 3 states). Leadership (n=35) and staff (n=289) were asked Pawtucket, Rhode Island.
to rate the implementation leadership (IL) and implementation cli- Results
mate (IC) of their site. Alignment and directionality of alignment (i.e., This presentation will include:
a positive relationship between leadership and staff ratings, or nega- 1) A brief explanation of the LIFT Together System (processes, com-
tive misalignment) between these two groups was calculated as less ponents, and outcomes).
than a half standard deviation in the difference between mean 2) A description of the lessons learned and challenges faced in the
scores [2-3]. Fidelity assessments for two universal behavioral health implementation of LIFT Together from the perspectives of the school,
prevention programs were then conducted, and further analysis will collaborating providers, and researchers. Examples include develop-
determine if alignment directionality affects implementation fidelity. ing a common understanding of the goals and processes, establish-
Results ing protocols for access to the critical populations, and
A significant proportion of sites reported negative misalignment be- implementing practices for school engagement.
tween leadership and staff. 37.1% of sites reported negative mis- 3) A description of data collection processes for the evaluation of
alignment on overall perceptions of IL, and 51.4% reported negative LIFT Together implementation and goals (e.g., integrating school and
misalignment on overall perceptions of IC. Additionally, negative mis- service provider data to evaluate outcomes).
alignment across the seven IL subscales (i.e., proactive, Conclusion
knowledgeable, supportive, perseverant, communication, vision, and This presentation provides an illustration of implementation practice
availability) ranged from 25.7% to 51.4%, and negative misalignment in real-world settings in the delivery and evaluation of a systematic,
across the seven IC subscales (i.e., focus, rewards, use of data, inte- community-based system for mobilizing schools to address student
gration, existing support, recognition, and educational support) concerns with a tiered package of school- and family-based pro-
ranged from 34.3% to 60.0%. grams. This presentation will not only illuminate the specific nature
of LIFT Together, but it also will elucidate contextual factors for local schools recruited increasing from 6/month prior to coach involve-
success and highlight the types of expertise and knowledge needed ment to 39/month after. Further, discussion with SPs about their con-
to facilitate local partnerships and implement initiatives that benefit cerns regarding participation also proved helpful, as most were
vulnerable youth and families in community settings. related to implementation of CBT in their work, rather than study
participation. Discussing the experiences of past program partici-
References pants, as well as program flexibility, helped assuage these concerns.
1. Thompson RW, Daly DL. The Family Home Program: An adaptation of Notably, SPs were generally not persuaded by discussion of study in-
the Teaching-Family Model at Boys Town. In: Whittaker JK, Del Valle JF, centives ($330 over 18 months), but rather by empirical evidence re-
Holmes L, editors. Therapeutic residential care with children and youth: lated to student mental health improvement.
developing evidence-based international practice London and Philadel- Conclusions
phia: Kingsley Publishers; 2015. p. 113- 123. Engagement of community members and personalized recruitment
2. Mason WA, Cogua JE, Thompson RW. Turning a big ship: unleashing the efforts were necessary to overcome barriers to study participation
power of prevention within treatment settings. J Soc Social Work Res. among schools. This resulted in engaging SPs from diverse school
2018; 9(4): 765-781. https://doi.org/10.1086/700847 settings and exceeding recruitment targets.
3. Oliver, RM, Lambert MC, Mason WA. A pilot study for improving
classroom systems within schoolwide positive behavior support. J Emot References
Behav Disord. 2019; 27(1):25–36. https://doi.org/10.1177/ 1. Kilbourne AM, Smith SN, Choi SY, Koschmann E, Liebrecht C, Rusch A,
1063426617733718 Abelson JL, Eisenberg D, Himle JA, Fitzgerald K, Almirall D. Adaptive
4. Mason WA, January S-A., Fleming CB, Thompson RW, Parra GR, Haggerty school-based Implementation of CBT (ASIC): clustered-SMART for building
KP, Snyder JJ. Parent training to reduce problem behaviors over the tran- an optimized adaptive implementation intervention to improve uptake
sition to high school: Tests of indirect effects through improved emotion of mental health interventions in schools. Implement Sci. 2018;13(1):119.
regulation skills. Child Youth Serv Rev. 2016;61:176-183. 2. Weiner BJ. A theory of organizational readiness for change. Implement
5. Anderson L, Ringle JL, Ross JR, Ingram SD, Thompson RW. Care Sci. 2009;4.1:67.
coordination services: a description of an alternative service model for at- 3. Glanz K, Bishop DB. The role of behavioral science theory in
risk families. J Evid Inf Soc Work. 2017;14:217-228. http:doi.org/ 10.1080/ development and implementation of public health interventions. Ann
23761407.2017.1306731 Rev Public health. 2010;31:399-418.
6. Patwardhan I, Duppong Hurley K, Thompson RW, Mason WA, Ringle JL.
Child maltreatment as a function of cumulative family risk: Findings from
the intensive family preservation program. Child Abuse Negl. 2017; A134
70:92–99. http://doi.org/ 10.1016/j.chiabu.2017.06.010 The role of outer context factors in the state-wide implementation
of evidence-based practices for students with Autism Spectrum
Disorder
A133 Allison Nahmias1, Melina Melgarejo2, Patricia Schetter1, Jessica
Engaging underserved communities in implementation research: Suhrheinrich2, Jennica Li1, Shaun Jackson1, Aubyn Stahmer1
1
strategies for success in the Adaptive School-based UC Davis MIND Institute, Sacramento, CA, USA; 2San Diego State
Implementation of CBT (ASIC) Trial University, San Diego, CA, USA
Amy Rusch, Jennifer Vichich, Kristen Miner, Seoyoun Choi, Michael Correspondence: Allison Nahmias ([email protected])
Prisbe, Elizabeth Koschmann, Celeste Liebrecht, Amy Kilbourne, Shawna Implementation Science 2020, 15(Suppl 2):A134
Smith
University of Michigan, Ann Arbor, MI, USA Background
Correspondence: Amy Rusch ([email protected]) Although evidence-based practices (EBPs) for children with Autism
Implementation Science 2020, 15(Suppl 2):A133 Spectrum Disorder (ASD) exist, there are significant challenges with
implementing these interventions in community settings. The Califor-
Background nia Autism Professional Training and Information Network (CAPTAIN)
Implementation studies are often criticized for engaging only early is a statewide cross-agency implementation team with the goal of
adopter sites, thus limiting study generalizability. Better implementa- scaling up use of EBPs for ASD using train-the-trainer methodology.
tion science requires understanding optimal tactics for engaging Data on model effectiveness and mechanisms of action are limited.
stakeholders in implementation research. Adaptive School-based Im- This project investigates the role of outer context factors
plementation of CBT (ASIC) is a large-scale randomized trial designed (organizational culture, leadership, structure, and resources) on im-
to test different implementation strategies to support school profes- plementation team member training performance using the EPIS [1]
sional (SP) delivery of cognitive-behavioral therapy (CBT) in high Framework.
schools across Michigan [1]. We analyzed methods used to recruit Materials and Methods
SPs at more than 100 diverse schools for ASIC participation and de- 101 directors from 87 Special Education Local Plan Areas (SELPAs, or-
scribe successful strategies used in this large-scale implementation ganizations that facilitate Special Education services in California)
study across different settings [2] and stages of change [3]. completed the Implementation Climate Scale (ICS), Implementation
Materials and Methods Leadership Scale (ILS), and ASD EBP Resource Assessment Tool. 194
Schools were recruited to ASIC over a 6-month period. A post-hoc CAPTAIN members reported on the frequency and quality of training
process evaluation of recruitment was conducted. Metrics collected in- and coaching. Generalized Estimating Equations were used to exam-
clude quantitative measures (e.g., number of attempts) and qualitative ine differences in implementation climate, leadership, and resources
feedback from recruiters on successful recruitment strategies. Following in regards to SELPA structure and size and their association with
recruitment, data were analyzed to identify patterns in successful recruit- CAPTAIN member performance.
ment efforts and codify effective strategies for recruiting SPs to ASIC. Results
Results ICS, ILS, and ASD EBP Resource scores varied by SELPA size. Large
With a goal of recruiting 100 schools, ASIC reached out to 272 SELPA directors reported better implementation climate in regards to
schools identified as candidates and ultimately, SPs at 114 schools focus on EBPs and existing supports to deliver EBPs than small SELP
were recruited over 6 months. The average SP required 5 contacts As, and better educational support for EBPs compared to medium
before agreeing to participate (range: 1-16). Following early low re- SELPAs (p-values < .05). Large SELPA directors also reported higher
cruitment numbers, the study team mobilized seven clinicians and proactive, supportive and perseverant implementation leadership
research assistants with mental health service experience, as well as than small SELPAs, and higher knowledgeable and proactive imple-
members of a statewide CBT coaching network to reach out to mentation leadership than medium SELPAs (p-values < .05). Large
schools. Leveraging these existing community partnerships served to SELPAs also reported greater partnerships with community stake-
significantly increase recruitment success, with average number of holders related to ASD EBP use than small SELPAs (p = .01). ICS
scores varied by whether the SELPA consisted of one school district References
(single) or multiple school districts (multi), with multi-district SELPAs 1. Odom SL, Collet-Klingenberg L, Rogers SJ, Hatton DD. Evidence-based
reporting higher selection for EBPs and selection for openness than practices in interventions for children and youth with autism spectrum
single-district SELPAs (p-values < .02). Proactive leadership was a sig- disorders. Prev Sch Fail. 2010;54(4):275–282.
nificant predictor of CAPTAIN performance (B = 3.77, p = .04). 2. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC.
Conclusions Fostering implementation of health services research findings into
Implementation leadership and climate vary across organizations practice: a consolidated framework for advancing implementation
suggesting variability more broadly. Proactive leadership relates to science. Implement Sci. 2009; 4:50.
frequency and quality of EBP training and coaching in schools. 3. Stahmer AC, Collings NM, Palinkas LA. Early intervention practices for
Matching targeted implementation efforts to context and children with autism: Descriptions from community providers. Focus
organizational functioning will be discussed. Autism Other Dev Disabl. 2005;20(2):66–79.
Reference
1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of A136
evidence-based practice implementation in public service sectors. Adm Graduate school training in structured cognitive behavioral
Policy Ment Health. 2010;38(1):4-23. doi:10.1007/s10488-010-0327-7. therapy protocols predicts greater evidence based psychotherapy
reach
Jiyoung Song1, Hector Garcia2, Erin Finley3, Shannon Wiltsey Stirman4
1
A135 National Center for PTSD, VA Palo Alto Health Care System, Palo Alto,
Do student characteristics affect teachers’ decisions to use 1:1 CA, USA; 2VA Texas Valley Coastal Bend Health Care System, Harlingen,
instruction? TX, USA; University of Texas Health Science Center San Antonio, San
Heather Nuske1, Melanie Pellecchia1, Viktor Lushin1, Keiran Rump1, Max Antonio, TX, USA; 4Stanford University, Stanford, CA, USA
Seidman1, Rachel Ouellette2, Diana Cooney1, Brenna Correspondence: Jiyoung Song ([email protected])
Maddox1, Gwendolyn Lawson1, Amber Song1, Erica Reisinger1, David Implementation Science 2020, 15(Suppl 2):A136
Mandell1
1
University of Pennsylvania, Philadelphia, PA, USA; 2Florida International Background
University, Miami, FL, USA The Veterans Health Administration recommends that patients with
Correspondence: Heather Nuske ([email protected]) PTSD receive either of the two evidence-based psychotherapies
Implementation Science 2020, 15(Suppl 2):A135 (EBPs), Cognitive Processing Therapy (CPT) and Prolonged Exposure
(PE). However, in one survey, clinicians who completed the national
Background PE training program reported that they only treated one or two pa-
One-to-one instruction is a critical component of evidence-based prac- tients at a time with PE [1]. In another survey, 69% of the clinicians
tices (EBPs) for students with autism spectrum disorder (ASD) [1], but is who were trained in CPT provided CPT “rarely” or “less than half the
not used as often as recommended. As described in the Consolidated time” [2]. Because underutiliziation of the EBPs leads to fewer pa-
Framework for Implementation Research [2], an important outer setting tients receiving the optimal treatments, it is important to identify
characteristic when considering EBPs is clients’ needs based on their and remediate clinician-level barriers that lead to low reach.
characteristics. Indeed, student characteristics may affect teachers’ deci- Materials and Methods
sions to select a treatment and/or implement one-to-one instruction In our current study, we surveyed clinicians across the United States
[3]. This study examined whether teachers’ reported use of one-to-one who primarily work on PTSD clinical teams (PCTs; [3]). They reported
discrete trial training (DTT) and pivotal response training (PRT) was as- whether they received graduate school training (GST) in structured
sociated with students’ clinical and demographic characteristics. cognitive behavioral therapy (CBT) protocols and rated agreement to
Materials and Methods the following clinician-level barriers to EBPs: discomfort of exposing
Participants were kindergarten-through-second-grade autism support patients to distress during PE and concerns of patients’ difficulty un-
teachers (n=80) and children aged 5-9 years with ASD (n=228). All derstanding CPT. We conducted mediation analyses with bootstrap-
teachers received training and consultation in the EBPs, and rated ping between GST, EBP barriers, and EBP usage.
children in several symptom domains using the Pervasive Develop- Results
mental Disorders Behavior Inventory. Children were assessed on cog- We found that GST (b = -0.27, t(222) = -2.08, p = .04) led to lower discomfort
nitive and language abilities using the Differential Abilities Scales, of exposing patients to distress during PE, and in turn, lower discomfort (b =
and on self-regulation difficulties using the Behavioral Interference -6.84, t(222) = -4.83, p < .001) predicted greater PE usage. GST and PE usage
Coding Scheme. Each month, teachers reported on their use of two had a significant average causal mediation effect (b = 1.83, 95% CI [0.00,
EBPs with each student during the past week. 4.10], p = .05). Because GST (b = 0.13, t(222) = 0.82, p = .41) was not signifi-
Results cantly associated with the CPT barrier, there was no statistical ground to test
Children’s higher sensory symptoms, lower social approach, lower for the mediation between GST, CPT barrier, and CPT usage.
verbal skills and higher self-regulation difficulties were associated Conclusions
with more frequent 1:1 DTT and PRT; significant child symptom do- Our results indicate that clinicians who used structured CBT protocols
mains each explained 8-15% of the variance in reported receipt of as part of their graduate school training are less likely to shy away
treatment. Children’s age, sex and race were not statistically signifi- from an EBP even when it might ask them of discomfort of exposing
cant predictors of children’s receipt of these EBPs. their patients to distress. To increase EBP reach, policy makers should
Conclusions promote for the inclusion of using structured CBT protocols in gradu-
These results provide an example of a situation where client charac- ate school curriculums.
teristics seem to influence providers’ use of EBPs. More obviously im-
paired students received more of each EBP. The findings beg the References
questions of whether teachers are accurate in their decisions regard- 1. Ruzek JI, Eftekhari A, Crowley J, Kuhn E, Karlin BE, Rosen CS. Post-training
ing who benefits from 1:1 instruction, and whether children should beliefs, intentions, and use of prolonged exposure therapy by clinicians
be matched to specific types of 1:1 instruction based on their clinical in the Veterans Health Administration. Adm Policy Ment Health.
characteristics. To the extent that experts think that less obviously 2017;44(1):123-132.
impaired students would benefit from 1:1 instruction, those working 2. Chard KM. Cognitive processing therapy train-the-trainer workshop. Cin-
with teachers should address practical, attitudinal and structural bar- cinnati, OH. 2014.
riers to providing 1:1 instruction to a larger proportion of students. 3. Garcia HA, DeBeer BR, Mignogna J, Finley EP. Treatments Veterans Health
This study highlights the importance of considering client character- Administration PTSD specialty program providers report their patients
istics in the development and study of implementation strategies, prefer: The role of training and theoretical orientation. Psychol Trauma.
across EBPs and contexts. 2019;11(8):837–841. https://doi.org/10.1037/tra0000442
A137 A138
Creating academic and organizational synergy within public Meta-analysis of implementation strategy effectiveness on general
education to support statewide scale up of EBP for students with education teacher adherence to evidence-based practices
Autism Spectrum Disorder James Merle, Clayton Cook, Andrew Thayer, Madeline Larson, Sydney
Jessica Suhrheinrich1,2, Patricia Schetter3,4, Ann England4,5, Melina Pauling, Jenna McGinnis
Melgarejo1, Allison Nahmias3, Aubyn Stahmer3 University of Minnesota, Minneapolis, MN, USA
1
San Diego State University, San Diego, CA, USA; 2Child and Adolescent Correspondence: James Merle ([email protected])
Services Research Center San Diego, CA, USA; 3UC Davis MIND Institute, Implementation Science 2020, 15(Suppl 2):A138
Sacramento, CA, USA; 4California Autism Professional Training and
Information Network, CA, USA; 5CA Department of Education, Background
Sacramento, CA, USA Intervention research in education science has produced a plethora of
Correspondence: Jessica Suhrheinrich ([email protected]) evidence-based practices (EBP) to improve student social, emotional, and
Implementation Science 2020, 15(Suppl 2):A137 behavioral (SEB) outcomes [1]. To ensure students benefit from these
practices, implementation strategies (techniques and methods to improve
Background implementation outcomes) [2], have been developed to bolster school-
Nationwide, 616,234 students were served for Autism Spectrum Dis- based practitioners’ EBP treatment integrity. These include action plan-
order (ASD) during 2017-18, an increase of 55% from 2007-08 [1]. Al- ning, prompts and reminders, coaching, and performance feedback [3-6].
though evidence-based practices (EBPs) for individuals with ASD However, because these strategies are often delivered simultaneously,
exist [2], use in community settings is limited. The California Autism mechanisms by which they produce effects remains unknown, hindering
Professional Training and Information Network (CAPTAIN) is a state- further understanding of causal relationships and efficient service delivery.
wide cross-agency collaboration with the goal of scaling up use of Therefore, the purpose of this study was to conduct a meta-analysis cat-
EBPs for ASD. CAPTAIN has over 400 members representing 140 egorizing and analyzing the effectiveness of discrete school-based imple-
school and community agencies who commit to training, coaching mentation strategies across service provision levels to inform limited
and engaging in collaboration. school resource allocation and identify future directions for research.
Methods and Materials Materials and Methods
CAPTAIN began as a clinical initiative 6 years ago, then further devel- Published studies of strategies used to increase teacher implementation
oped under the influence of implementation science methodology. of SEB EBPs were included. Effect sizes were calculated, and a robust vari-
The Exploration, Planning, Implementation and Sustainment frame- ance estimation meta-regression model (RVE) was used to hierarchically
work (EPIS) [3] has impacted targeted strategy use for statewide scale analyze average effects and conduct moderator analyses [7]. Funnel plots
up of EBPs by informing the development of key partnerships, imple- and Egger’s test were used to assess publication bias [8].
mentation goals, and collaborative processes within CAPTAIN. Results
Results Preliminary results of 31 single-subject studies indicate that active-
The panel will highlight how education policy and implementation data implementation strategies targeting performance deficits were effect-
have influenced CAPTAIN practices and procedures within a ive overall for increasing teacher adherence to EBPs above baseline
community-academic partnership. These factors will be presented by and pre-implementation training alone (Hedge’s g = 2.45). Mainten-
purveyors, intermediaries and implementation science researchers. The ance strategies, such as dynamic fading of supports, while sparse, in-
founding co-coordinator of CAPTAIN and a purveyor of this initiative dicated effective sustainment of implementation behavior (g = 0.8).
will share information about the policies that have influenced the de- Performance-feedback was the most common strategy (n = 23; 74%),
velopment of the network and how key partnerships have been though preliminary results indicate it may not be the most effective
formed. A fellow founding co-coordinator of CAPTAIN that serves as an strategy across all intervention tiers. Twenty group-design studies
intermediary in the CAPTAIN project will share how implementation sci- were included and analyses are underway.
ence and the EPIS Model have informed the CAPTAIN implementation Conclusions
goals and procedures and how this information has been shared with Teacher treatment integrity improved with added supports; however,
state and local agencies through continuous improvement cycles. The once supports were removed, implementation decreased in over
research director for CAPTAIN will present current funded research ini- 40% of the studies that collected follow-up data. As is true in the
tiatives involving CAPTAIN and will facilitate discussion. greater implementation literature, [9], further methods for sustaining
Conclusions implementation are needed. Gradual fading reduced implementer
Mixed-methods data will be presented, informed by an internal sur- drift, and practitioners should consider it with active-implementation
vey of CAPTAIN members (n=414), a statewide survey of educational supports. This study contributes to the broader implementation lit-
professionals and administrators (n=1700), and qualitative focus erature by providing discrete strategy effectiveness of practitioner-
group outcomes from providers and CAPTAIN members (n=30). level performance-based implementation strategies and informs fu-
A subset of outcomes will be presented with a focus on multiple per- ture research categorizing and analyzing implementation strategies
spectives on barriers, implementation leadership and implementation within existing frameworks [2].
climate, implementation strategy use, and the distribution of decision
making across organizational levels.
References
1. Owens JS, Lyon AR, Brandt NE, Warner CM, Nadeem E, Spiel C, Wagner
References M. Implementation science in school mental health: key constructs in a
1. U.S. Department of Education, EDFacts Data Warehouse (EDW): IDEA Part developing research agenda. School Ment Health. 2014;6(2):99-111.
B Child Count and Educational Environments Collection, file 2. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM,
specifications 002 and 089. 2017-2018. https://www2.ed.gov/ programs/ Proctor EK, Kirchner JE. A refined compilation of implementation strategies:
osepidea/618-data/static-tables/2017-2018/part-b/child-count-and-educa- results from the Expert Recommendations for Implementing Change (ERIC)
tional-environment/1718-bchildcountandedenvironment-18.xlsx. project. Implement Sci. 2015;10(1):21. doi:10.1186/s13012-015-0209-1.
Accessed 11 July 2018 3. Bethune KS. Effects of coaching on teachers’ implementation of tier 1
2. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, Brock ME, school-wide positive behavioral interventions and support strategies. J
Plavnick JB, Fleury VP, Schultz TR. Evidence-based practices for children, Posit Behav Interv. 2017;19(3):131-142.
youth, and young adults with autism spectrum disorder: a comprehen- 4. Collier-Meek MA, Fallon LM, Defouw ER. Toward feasible implementation
sive review. J Autism Dev Disord. 2015;45(7):1951-1966. doi:10.1007/ support: E-mailed prompts to promote teachers’ treatment integrity.
s10803-014-2351-z. School Psychol Rev. 2017;46:379-394.
3. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of 5. Collier-Meek MA, Sanetti LM, Boyle AM. Providing feasible implementa-
evidence-based practice implementation in public service sectors. Adm tion support: direct training and implementation planning in consult-
Policy Ment Health. 2010;38(1):4-23. doi:10.1007/s10488-010-0327-7. ation. School Psychol Forum. 2016;10.1:106-119.
6. Sanetti LMH, Collier-Meek MA, Long ACJ, Byron J, Kratochwill TR. Increas- 3. Stormont M, Reinke W, Herman K. Teachers’ characteristics and ratings for
ing teacher treatment integrity of behavior support plans through con- evidence-based behavioral interventions. Behav Disorders. 2011;37:19-29.
sultation and implementation planning. J School Psychol. 2015;53: 209- 4. Dagenais C, Lysenko L, Abrami PC, Bernard RM, Ramde J, Janosz M. Use
229. of research-based information by school practitioners and determinants
7. Hedges LV, Tipton E Johnson MC. Robust variance estimation in meta‐ of use: a review of empirical research. Evid Policy. 2012;8:285-309.
regression with dependent effect size estimates. Res Synth Method.
2010;1:39-65. doi:10.1002/jrsm.5
8. Egger M, Davey Smith G, Schneider M, Minder, C. Bias in meta-analysis A140
detected by a simple, graphical test. BMJ. 1997;315(7109):629-34. Extending implementation science to the rural school setting
9. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic Benjamin Ingman1,2, Elaine Belansky1,2
1
review of the Exploration, Preparation, Implementation, Sustainment Center for Rural School Health & Education, Denver, CO, USA;
2
(EPIS) framework. Implement Sci. 2019; 14:1. University of Denver, Denver, CO, USA
Correspondence: Benjamin Ingman ([email protected])
A139
Ready, willing, and able? exploring education researcher Background
engagement in dissemination Children in rural America face unique health disparities compared to
Taylor Koriakin, Sandra Chafouleas, Lisa Sanetti, Jennifer Dineen their urban counterparts, including higher substance use; sexual ac-
University of Connecticut, Storrs, CT, USA tivity and teen pregnancy; and suicide rates [1-3]. While there are nu-
Correspondence: Taylor Koriakin ([email protected]) merous evidence-based practices (EBPs) K-12 schools can implement
Implementation Science 2020, 15(Suppl 2):A139 to promote students’ physical, social-emotional, and academic well-
being [4-11], little is known about the contextual factors that facili-
Background tate or inhibit rural school districts’ selection of EBPs.
Getting Research into Policy and Practice [1] (GRIPP) is a continued Our center is currently facilitating 21 high-poverty, rural school dis-
concern in education with several studies citing limited knowledge tricts through a strategic planning process to create comprehensive
and use of evidence-based practices (EBP) by school practitioners [2, health and wellness plans that include EBPs for physical and mental
3]. Although studies exist evaluating practitioners’ perspectives on health. The process is completed by a school district task force com-
GRIPP [4], there are presently no studies examining researchers’ ap- prised of school administrators, teachers, students, parents, and com-
proaches to dissemination. Given that they generate the evidence- munity members under the guidance of an external facilitator.
base for school-based practice, an understanding of how researchers In this study, we adapted the Consolidated Framework for Implemen-
communicate their findings is an important piece in understanding tation Research (CFIR)[12] to the rural school context to understand
implementation of EBPs. The purpose of the present study was to the contextual factors associated with school districts’ selection of
understand education researchers’ engagement in dissemination ac- EBPs. We explore (1) the EBPs selected by these rural school districts,
tivities targeting non-research audiences. (2) the prevalence of contextual factors amongst districts, and (3) the
Materials and Methods association of contextual factors with EBPs selected.
School psychology and special education researchers (n = 226) at Re- Materials and Methods
search Intensive institutions completed on online survey related to We have developed (and are currently administering) an 81 item sur-
their dissemination practices. Respondents answered items pertain- vey to evaluate the contextual factors of implementation for this ini-
ing to their most frequently used dissemination modalities, target au- tiative. This pen and paper survey, which was designed for rural
diences, barriers, and time dedicated to dissemination. Participants schools completing the planning process, accounts for all 39 CFIR
were also asked to rank which dissemination activities they perceived constructs. The survey is administered at the end of the final meeting
as having the greatest impact on education practice. of the process and completed by members of the 21 rural school dis-
Results trict task forces. All survey data will be collected by May 2019.
Over half of sample (59.9%) reported spending less than two hours per Results
week on dissemination activities targeting non-research audiences. Partic- Results will reveal the prevalence and relationships of contextual factors
ipants were asked to rank order their most frequently used dissemination and EBPs selected by rural school districts as a result of the process.
practices and indicated that academic journal articles (rated as primary Conclusions
dissemination activity by 63% of sample) and conference presentations These results will be instructive concerning the importance of particular
(rated primary activity by 15%) were the most frequently used modalities contextual factors as they apply to the selection of EBPs in rural
for dissemination. Although participants reported that they felt that schools. Additionally, the survey developed and administered provides
professional development sessions, meetings with stakeholders, and an example of evaluating CFIR constructs in K-12 school settings, which
practitioner-focused books have the greatest impact on educational prac- contributes to the growing SIRC instrument review project.
tice, they were not as frequently utilized by respondents as peer-
reviewed articles and conference sessions. Common barriers reported by References
respondents were limited time to dedicate to dissemination (rated as pri- 1. Atav S, Spencer G. Health risk behaviors among adolescents attending
mary barrier by 37% of sample) and that dissemination is a low priority rural, suburban, and urban schools: a comparative study. Fam
at institutions (rated as primary barrier by 20%). Community Health. 2002;25:53-64.
Conclusions 2. Lambert D, Gale JA, Hartley D. Substance abuse by youth and young
Overall, respondents reported engaging in low rates of dissemination adults in rural America. J Rural Health. 2008; 24:221-8.
targeting applied, non-research audiences. With limited time, educa- 3. Health Management Associates. Community conversations to inform youth
tion researchers appear to focus their resources on activities associ- suicide prevention. 2018: https://coag.gov/sites/default/files/
ated with promotion and tenure (e.g., publishing peer-reviewed final_youth_suicide_in_colorado_report_10.01.18.pdf. Accessed 18 March 2019.
journal articles). Although participants valued the importance of dis- 4. Botvin GJ, Griffin KW. Life skills training: Empirical findings and future
semination targeting those in applied settings, they reported that directions. J Prim Prev. 2004; 25:211-232.
these activities are largely not valued by their institutions. 5. Caldarella P, Shatzer RH, Gray KM, Young KR, Young EL. The effects of
school-wide positive behavior support on middle school climate and stu-
References dent outcomes. Res Mid-Level Educ. 2011;35:1-14.
1. Hoover SA. When we know better, we don’t always do better: facilitating 6. Celio CI, Durlak J, Dymnicki A. A meta-analysis of the impact of service-
the research to practice and policy gap in school mental health. Sch learning on students. J Experiential Education. 2011;34:164-181.
Ment Health. 2018;10(2):190-198. 7. Durant N, Harris SK, Doyle S, Person S, Saelens BE, Kerr J, Norman GJ,
2. Burns MK, Ysseldyke JE. Reported prevalence of evidence-based instruc- Sallis JF. Relation of school environment and policy to adolescent
tional practices in special education. J Spec Educ. 2009;43:3-11. physical activity. J Sch Health. 2009;79:153-159.
8. Evans A, Ranjit N, Rutledge R, Medina J, Jennings R, Smiley A, Hoelscher A142

D. Exposure to multiple components of a garden-based intervention for A multi-component home visitation program to prevent child
middle school students increases fruit and vegetable consumption. maltreatment: effects on parenting and child functioning
Health Promot Pract. 2012;13:608-616. Elizabeth Demeusy, Jody Manly, Robin Sturm, Sheree Toth
9. Hadlaczky G, Hökby S, Mkrtchian A, Carli V, Wasserman D. Mental Health Mt. Hope Family Center, University of Rochester, Rochester, NY, USA
First Aid is an effective public health intervention for improving Correspondence: Elizabeth Demeusy ([email protected])
knowledge, attitudes, and behaviour: a meta-analysis. Int Rev Psychiatry. Implementation Science 2020, 15(Suppl 2):A142
2014;26:467-475.
10. Kelly CM, Mithen JM, Fischer JA, Kitchener BA, Jorm AF, Lowe A, Scanlan Background
C. Youth mental health first aid: a description of the program and an No single approach can meet the multi-dimensional needs of impoverished,
initial evaluation. Int J Ment Health Syst. 2011;5:1-9. high-risk families. With this in mind, the Building Healthy Children (BHC) inter-
11. Kirby D, Laris BA. Effective curriculum based sex and STD/HIV education vention program was designed as a collaborative community initiative to pre-
programs for adolescents. Child Dev Perspect. 2009;3:21-29. vent child maltreatment and support healthy development in newborns of
12. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC. young mothers [1-2]. This inter-agency collaborative was comprised of med-
Fostering implementation of health services research findings into ical, university and community partners, with a strong focus exporting
practice: a consolidated framework for advancing implementation evidence-based models and translating research to practice. BHC was de-
science. Implement Sci. 2009; 4:50. signed as a home visitation program that employed a tiered model of service
delivery based on families’ individual needs, in order to effectively and effi-
ciently deliver outreach and evidence-based treatment models. These treat-
A141 ment models addressed parenting (Parents as Teachers), attachment (Child-
Community mental health providers’ use of parent training with Parent Psychotherapy), and maternal depression (Interpersonal Psychotherapy
Medicaid-enrolled families of children with autism for Depression), and each has received substantial evidentiary support [3-6].
Diondra Straiton, Brooke Ingersoll Materials and Methods
Michigan State University, East Lansing, MI, USA The current study utilizes a longitudinal follow-up design, to examine the ef-
Correspondence: Diondra Straiton ([email protected]) fects of BHC on parenting and child behavior once the child is in elementary
Implementation Science 2020, 15(Suppl 2):A141 school (6-10 years old). The anticipated sample size is 100 children (45% male)
and their biological mothers (baseline age, M=19). In addition, data is also be-
Background ing collected from the child’s teacher regarding child behavior in the class-
Evidence-based parent training, in which providers systematically room in order to provide a multi-informant perspective across settings. Thus
train parents to implement specific intervention strategies with their far, data has been collected from approximately 50 families. Data collection is
child, is an underutilized treatment for children with autism spectrum scheduled to conclude by 7/1/19.
disorder (ASD) [1]. In 2012, Michigan passed the Medicaid Autism Results
Benefit for Behavioral Health Treatment, which provides funds for This presentation will examine the long-term effects of BHC on child mal-
Medicaid-enrolled children with ASD for applied behavior analysis treatment, out-of-home placement and harsh and inconsistent parenting,
(ABA) services, including parent training. However, a review of billing as well as positive parenting practices. In addition, it will examine the ef-
data shows that few parent training sessions have been billed under fects of BHC on child externalizing behavior and self-regulation.
the Medicaid Autism Benefit, despite providers having the ability to Conclusions
bill for the service at a high reimbursement rate. To our knowledge, The information gleaned from this study will help us to better understand
no other study to date has examined reasons for why parent training how to develop effective inter-agency partnerships in order to increase ac-
for ASD is underutilized within community mental health settings. cessibility to high-quality, evidence-based mental health services for vulner-
Materials and Methods able children and families. This effectiveness study provides information
This mixed-methods project examined the use of parent training during ABA about implementing efficacious interventions within existing community in-
under the Michigan Medicaid Autism Benefit for children with ASD under frastructure. Finally, the results of this study will help us to better understand
age 21. Descriptive statistics and multiple regression were used to analyze how to prevent violence against children, and to foster a safe and healthy
Medicaid claims data for 879 children and survey data from 97 ABA providers. caregiving environment for children to grow and thrive.
Content analysis was used to analyze open-ended survey items and thematic
analysis was used to analyze interviews from a subset of 13 providers. References
Results
1. Paradis HA, Sandler M, Manly JT, Valentine L. Building healthy children:
Results demonstrated that: a) frequency of parent training encoun-
Evidence-based home visitation integrated with pediatric medical homes.
ters was very low, b) ABA providers’ conceptualization of parent
Pediatrics. 2013;132(Suppl 2):S174-S179.
training is inconsistent with the literature; c) providers report using 2. Toth SL, Manly JT. Bridging research and practice: challenges and
evidence-based parent training strategies at a moderate-to-high level
successes in implementing evidence-based preventive intervention strat-
on the survey, but infrequently spontaneously mention those strat-
egies for child maltreatment. Child Abuse Negl. 2011;35(8):633-636.
egies in interviews; d) providers use sessions for other purposes; e)
3. Cicchetti D, Rogosch FA, Toth SL. Fostering secure attachment in infants in
providers report having limited related training experiences; f) and maltreating families through preventive interventions. Dev Psychopathol.
providers report numerous barriers and facilitators which are related
2006;18(03):623-649.
to their reported extensiveness of parent training.
4. Toth SL, Rogosch FA, Oshri A, Gravener-Davis J, Sturm R, Morgan-López AA.
Conclusions
The efficacy of interpersonal psychotherapy for depression among econom-
Barriers and facilitators were identified at the family-, provider-, and ically disadvantaged mothers. Dev Psychopathol. 2013;25(4pt1):1065-1078.
organization-levels and map well onto the Consolidated Framework
5. Wagner MM, Clayton SL. The Parents as Teachers program: results from
for Implementation Research, including outer setting (policy regard-
two demonstrations. Future Child. 1999; 9(1):91-115.
ing Medicaid Autism Benefit), inner setting (agency-level characteris-
6. Weissman MM, Markowitz JC, Klerman GL. Comprehensive guide to
tics), and the individuals involved (providers). Results from this study interpersonal psychotherapy. New York: Basic Books; 2000.
will be used to design an intervention to increase uptake of
evidence-based parent training in this system.
A143
Screening for perinatal depression and anxiety: the Appalachian
Reference provider’s perspective on current practice and barriers
1. Hume K, Bellini S, Pratt C. The Usage and perceived outcomes of early Mira Snider, Shari Steinman
intervention and early childhood programs for young children with West Virginia University, Morgantown, WV, USA
Autism Spectrum Disorder. Top Early Child Spec Educ. 2005;25(4):195-207. Correspondence: Mira Snider ([email protected])
doi:10.1177/ 02711214050250040101 Implementation Science 2020, 15(Suppl 2):A143
Background case study included staff and administrators from partner organiza-
Postpartum Anxiety (PPA) and Postpartum Depression (PPD) are tions, members of the research team, local community members,
common mental health issues that go largely undiagnosed and un- and Alma peer mentors. All key informants (N=15) completed a sur-
treated in the United States [1]. There are no federal policies requir- vey that assessed the organizational and contextual factors known to
ing screening of new mothers, and only thirteen states have passed influence the implementation of programs, developed using the
legislature or convened task forces to promote screening for PPD in Practical, Robust, Implementation and Sustainability Model (PRISM)
perinatal care. Similar efforts have not been made to promote PPA [2]. All key informants were given the opportunity to participate in
screening [2]. Although screening is recognized as a valuable practice individual interviews or focus groups to explore qualitatively factors
that may increase new mothers’ access to mental health care, there associated with the successful implementation of Alma, and to
is limited data available on how this legislature has translated to follow-up on findings from the survey.
real-world screening and referral practice. The current study exam- Results
ines current screening and referral practices of perinatal providers in We will identify key facilitators and barriers that were identified at
West Virginia – a state that has taken legislative action to promote the following contextual levels: broader community/environment,
depression screening in perinatal women. Barriers to screening and organization, program, participant. For example, at the broader com-
referral that could be targeted by future implementation or advocacy munity/environment level, the most commonly reported facilitator
efforts are also explored. for Alma program implementation (92.3% of the sample) was, “a
Materials and Methods need for mental health resources in the community and the most
Approximately 50 perinatal providers in urban and rural settings commonly reported barrier (38.5% of the sample) was, “poor commu-
across West Virginia will be recruited to complete an online survey. nication streams in the community.”
Current screening and referral practices for PPA and PPD, perspec- Conclusions
tives on best screening practices, and barriers to discussing mental The organizational and program characteristics were identified to be
health issues with patients will be assessed. Providers’ perceived overall key facilitators for implementation of the Alma program in
feasibility, acceptability, norms, and intention to use screening tools this setting, whereas the broader community and environment con-
or referrals with perinatal women will also be examined. text was identified to be an overall barrier. Specific barriers and facili-
Results tators within each of these levels are identified, which could inform
Data collection for the current study is ongoing. Preliminary analyses broader implementation and spread of the program to new sites.
on a small subset of providers have indicated that obstetrician/gyne-
cologists in urban clinics screen a majority of patients for anxiety and References
depression symptoms, but providers do not consistently conduct 1. Chowdhary N, Sikander S, Atif N, Singh N, Ahmad I, Fuhr DC, Rahman A,
screening at each visit. Approximately 4-5% of providers’ caseloads Patel V. The content and delivery of psychological interventions for
were referred for mental health care in the past year, and referral perinatal depression by non-specialist health workers in low and middle
rates were higher for patients endorsing depression symptoms com- income countries: a systematic review. Best Pract Res Clin Obstet Gynae-
pared to patients endorsing anxiety symptoms. Providers reported col. 2014;28(1):113-33.
limited time as a barrier to screening, and a lack of consistent screen- 2. Feldstein AC, Glasgow RE. A practical, robust implementation and
ing as a barrier to providing mental health referrals to women. sustainability model (PRISM) for integrating research findings into
Conclusions practice. Jt Comm J Qual Patient Saf. 2008;34:228–43.
Perinatal providers in West Virginia screen for depression and anx-
iety; however, screening and referral may be inconsistent across dis-
orders. Having limited time to screen patients may interfere with A145
mandated screening in perinatal care. Fidelity to motivational interviewing and caregiver engagement in
the Family Check-Up 4 Health Program: longitudinal associations
References in a hybrid effectiveness-implementation trial
1. Accortt EE, Wong MS. It is time for routine screening for perinatal mood Cady Berkel1, JD Smith2, Anne Mauricio1, Jenna Rudo-Stern1, Liz
and anxiety disorders in obstetrics and gynecology settings. Obstet Alonso1, Sara Jimmerson1, Lizette Trejo1
1
Gynecol Surv. 2017;72(9):553-568. Arizona State University, Tempe, AZ, USA; 2Northwestern University
2. Rowan PJ, Duckett SA, Wang JE. State mandates regarding postpartum Feinberg School of Medicine, Chicago, IL, USA
depression. Psychiatr Servic. 2015; 66(3):324-328. Correspondence: Cady Berkel ([email protected])
A144 Background
Factors influencing implementation of the Alma program: This study examines fidelity to the Family Check-Up 4 Health (FCU4-
structured peer mentoring for depressed perinatal Spanish- Health) in a randomized hybrid trial evaluating effects on the preven-
speaking women tion of excess weight gain in pediatric primary care [1]. The program
Rachel Vanderkruik, Sona Dimidjian, Caitlin McKimmy includes a comprehensive assessment, followed by a feedback and
CU Boulder, Boulder, CO, USA motivation session, in which Motivational Interviewing (MI) is used to
Correspondence: Rachel Vanderkruik ([email protected]) engage families in follow-up parenting modules and community-
Implementation Science 2020, 15(Suppl 2):A144 based programs to address social determinants of health. The
COACH observational rating system assesses fidelity to the FCU4-
Background Health. In a prior trial, COACH ratings have been associated with
To address a critical gap in resources for depressed Latina pregnant higher concurrent ratings of parent engagement, and in turn, longi-
and early parenting women in rural settings, we co-designed and eval- tudinal improvements in parenting and child behaviors [2]. Further,
uated the Alma Program utilizing an innovative model of “task-sharing” coordinators with more training had better MI skills, and this was
[1], where peers are trained to mentor perinatal women experiencing positively linked with parent engagement [3] suggesting that MI
depression. In-depth examination of the extent to which contextual fac- skills are a core component of change. This study extends prior work
tors influence implementation outcomes has the potential to advance to advance the science of fidelity assessment when implementing a
understanding of the relationships among contextual adaptation, suc- preventive parenting program.
cess of a program, and dissemination to new settings. Materials and Methods
Materials and Methods The trial includes families (n=240) with children ages 6-12 years iden-
We conducted a case study of the Alma program to identify the tified in pediatric primary care with elevated body mass index (BMI)
organizational, contextual, and cultural factors that served as barriers for age and gender (≥85th percentile). 68% of parents were Latino
or facilitators to the implementation of the Alma program within a and 38% spoke Spanish. After completing the baseline assessment,
rural Colorado community. Key informants who participated in the families were randomized to the FCU4Health program (n=140) or
usual care (n=100). The FCU4Health involved 3 feedback sessions, Caregivers who had received mental health therapy were more likely
with parenting support sessions and referrals to community pro- to correctly define EBP (p<.01), and those whose teens had received
grams (e.g., nutrition, physical activity, social services, health/mental mental health therapy less strongly valued the outcome over the
health programs), over a 6-month period. process of therapy (p<.01). Multivariate analyses revealed that having
Results a teens with legal problems and substance use problems significantly
COACH coding of this sample is nearly complete. Analyses will be influenced how strongly caregivers preferred the term “therapy
completed in July 2019. We first plan to examine the temporal asso- based on evidence”. Caregivers whose teens had internalizing prob-
ciations between the FCU4Health coordinator’s use of MI in the 3 lems and legal problems less strongly favored proven treatment over
feedback sessions with caregiver engagement over time using a an individualized approach (ps<.01); only teen legal problems was
cross-lagged panel model. Second, using multiple regression, we will significant in the multivariate analysis.
examine associations between ratings of MI skills with families’ en- Conclusions
gagement in community-based programs, accounting for salient fam- Although most caregivers correctly defined and valued EBP, caregiver
ily demographic factors such as income, insurance, parent health, and teen treatment history and teen psychopathology moderated
and acculturation markers. this effect. Most notably, caregivers of teens with psychopathology
Conclusions symptoms and a therapy history were less likely to value principles
This study will inform the ways in which fidelity to MI is associated of EBP. Caregivers of teens with substance use and legal problems
with behavior change in an evidence-based parenting program—ad- also tended to prefer other terms over EBP.
ding valuable knowledge to the field that has implications for fidelity
monitoring, coordinator training, and program development. References
Trial Registration ClinicalTrials.gov NCT03013309 1. Becker SJ, Weeks BJ, Escobar KI, Moreno O, DeMarco CR, & Gresko SA.
Impressions of “Evidence-Based Practice”: a direct-to-consumer survey of
References caregivers concerned about adolescent substance use. Evid Based Pract
1. Smith JD, Berkel C, Jordan N, Atkins DC, Narayanan SS, Gallo C, Grimm KJ, Child Adolesc Ment Health. 2018; 3: 70–80. doi:10.1080/
Dishion TJ, Mauricio AM, Rudo-Stern J, Meachum MK, Winslow E, Bruen- 23794925.2018.1429228
ing MM. An individually tailored family-centered intervention for pediatric
obesity in primary care: study protocol of a randomized type II hybrid
implementation-effectiveness trial (Raising Healthy Children study). Imple- A147
ment Sci. 2018;13(11):1–15. doi:10.1186/s13012-017-0697-2 Implementing a child mental health intervention in child welfare:
2. Smith JD, Dishion TJ, Shaw DS, Wilson MN. Indirect effects of fidelity to CW staff perspectives on feasibility and acceptability
the Family Check-Up on changes in parenting and early childhood prob- Geetha Gopalan1, Kerry-Ann Lee2, Tricia Stephens1,3, Mary Acri4, Cole
lem behaviors. J Consult Clin Psychol. 2013;81(6):962–974. doi:10.1037/ Hooley5, Caterina Pisciotta4
1
a0033950 Hunter College, New York City, NY, USA; 2University of Maryland,
3. Smith JD, Rudo-Stern J, Dishion TJ, Stormshak EA, Montag S, Brown K, Ra- College Park, MD, USA; 3City University of New York, New York City, NY,
mos K, Shaw DS, Wilson MN. Effectiveness and efficiency of observation- USA; 4New York University School of Medicine, New York City, NY, USA;
5
ally assessing fidelity to a family-centered child intervention: a quasi- Brown School, Washington University, St. Louis, MO, USA
experimental study. J Clin Child Adolesc Psychol. 2019;48(1):16-28. Correspondence: Geetha Gopalan ([email protected])
doi:10.1080/15374416. 2018.1561295 Implementation Science 2020, 15(Suppl 2):A147
Background
A146 Children with behavior difficulties reared by child welfare (CW)-involved
Disseminating evidence-based treatments: teen psychopathology families often fail to receive needed mental health (MH) treatment due
and treatment history moderate caregiver perceptions of EBT to limited service capacity and chronic engagement difficulties [1].
Margaret Crane1, Sarah Helseth2, Kelli Scott2, Sara Becker2 Task-shifting strategies (World Health Organization, 2008) can increase
1
Temple University, Philadelphia, PA, USA; 2Brown University, Providence, MH treatment access by relocating treatment to alternative service plat-
RI, USA forms (e.g., CW service settings) and utilizing a non-specialized work-
Correspondence: Margaret Crane ([email protected]) force (e.g., CW caseworkers) to deliver MH interventions while
Implementation Science 2020, 15(Suppl 2):A146 supervised by trained clinicians. In this study, an evidence-based, mul-
tiple family group intervention (The 4Rs and 2Ss Program for Strength-
Background ening Families [4R2S]; [2-4]) to reduce child disruptive behavioral
To effectively promote evidence-based practice (EBP), it is important difficulties was modified so that it could be delivered by CW case-
to consider whether caregivers understand and view the concept fa- workers without advanced MH training. This study examines the de-
vorably. Previous research found that caregivers with lower educa- gree to which CW staff perceived delivering 4R2S in CW placement
tion, with lower SES, and from a minority racial group were less likely prevention services as feasible and acceptable, as well as those factors
to correctly define EBP and had a less favorable view of EBP [1]. This reported to facilitate or hinder feasibility and acceptability.
study examined how caregivers define, value, and prefer to describe Materials and Methods
EBP, and how responses varied based on caregiver and teen psycho- This mixed methods study collected quantitative and qualitative data
pathology and treatment history. from caseworkers (n=6), supervisors (n=4) and administrators (n=2).
Materials and Methods Quantitative and qualitative data focused on feasibility (e.g., participant
Caregivers (N=411; 86% female; 88% non-Hispanic Caucasian) con- flow, treatment fidelity, treatment attendance, CW staff perspectives on
cerned about their teen’s (age 12-19) substance use completed an on- feasibility and appropriateness) and acceptability (CW staff perspectives
line survey as part of a larger study. Caregivers selected the correct on acceptability and attitudes towards evidence-based practices). De-
definition of EBP, indicated their preference for describing EBP, and in- scriptive statistics compared quantitative data to pre-determined study
dicated whether they valued EBP treatment principles (proven vs. indi- benchmarks for high feasibility and acceptability. Qualitative data were
vidualized treatment; treatment process vs. treatment outcome). Chi- coded into relevant a priori (feasibility, acceptability) and emergent
square analyses evaluated caregiver responses by caregiver and teen themes. Mixed methods analytic strategies focused on integration at
treatment history, and teen mental health and substance use problems. the analysis stage, comparing quantitative and qualitative findings
Multivariate logistic regressions examined which variables were associ- side-by-side to identify points of convergence or expansion.
ated with the greatest likelihood of response selection. Results
Results Results indicate that CW staff perceived implementing a modified
Most caregivers correctly defined EBP, preferred the concept of “ther- version of the 4R2S in placement prevention services as generally
apy based on evidence”, preferred proven over individualized treat- feasible and acceptable, with some exceptions (e.g., inconsistent fam-
ment, and valued the outcome over the process of therapy. ily attendance). Factors facilitating feasibility and acceptability include
agency and logistical support, provider characteristics, 4R2S content & were not found between job engagement and self-efficacy (b = .04),
strength-based focus, 4R2S ease of use, and perceived benefits for fam- nor job engagement and job satisfaction (b = .02).
ilies and providers. Barriers to feasibility and acceptability included fam- Conclusions
ily characteristics, eligibility/recruitment procedures, logistical The results from the SEM analysis suggest that there are significant
challenges, as well as issues with external supervision. relationships between job engagement and HPC factors, which indi-
Conclusions cates that the HPC model is a valid tool for improving care quality in
Findings from these key stakeholders can inform similar efforts to im- Norwegian child welfare institutions. Future implementation of the
plement child MH EBPs within CW services. HPC model has significant implications for future policy develop-
ments, institutional practitioners, and implementation researchers.
References
1. Gopalan G, Chacko A, Franco LM, Dean-Assael KM, Rotko L, Marcus SM, References
Hoagwood KE, McKay M. Multiple Family Group service delivery model 1. Kayed NS, Jozefiak T, Rimehaug T, Tjelflaat T, Brubakk AB, Wichstrøm L. Psykisk
for youth with disruptive behaviors: child outcomes at 6-month follow- helse hos barn og unge i barnevernsinstitusjoner. Trondheim, Norway: NTNU
up. J Child Fam Stud. 2015;24(9):2721-2733. Regionalt Kunnskapssenter for Barn og Unge – Psykisk Helse og Barnevern; 2015
2. Chacko A, Gopalan G, Franco L, Dean-Assael K, Jackson J, Marcus S, Hoag- 2. Kylling R. En kvantitativ analyse av High Performance Cycle og en test
wood K, McKay M. Multiple Family Group service model for children with av jobbengasjement som et supplement til modellen, i barnevernsinstitusjoner
disruptive behavior disorders: child outcomes at post-treatment. J Emot ved Region-Nord. [master’s thesis]. Oslo, Norway: OsloMet University; 2014
Behav Disord. 2015;23(2):67-77. 3. Locke EA. Latham GP. New Developments in Goal Setting and Task
3. Gopalan G, Goldstein L, Klingenstein K, Sicher C, Blake C, McKay M. Performance. New York, NY: Routledge; 2013.
Engaging families into child mental health treatment: updates and
special considerations. J Can Acad Child Adolesc Psychiatry. 2010;19(3):
182-196. A149
4. Gopalan G. Feasibility of improving child behavioral health using task- Understanding rewards to facilitate implementation: perceptions
shifting to implement the 4Rs and 2Ss program for strengthening fam- of rewards and incentives across two government-supported
ilies in child welfare. Pilot and Feasibility Stud. 2016;2(21). doi 10.1186/ systems in Kenya
s40814-016-0062-2. Noah Triplett1, Grace Woodard1, Christine Gray2, Leah Lucid1, Prerna
Martin1, Rosemary Meza1, Kathryn Whetten2, Tyler Frederick1, Cyrilla
Amanya3, Augustine Wasonga3, Shannon Dorsey1
1
University of Washington, Seattle, WA, USA; 2Duke University, Durham,
A148 NC, USA; 3ACE Africa Kenya, Mumias, Kenya
Increasing quality of care in Norwegian child welfare institutions: a Correspondence: Noah Triplett ([email protected])
quantitative analysis of factors from the high performance cycle
and a test of job engagement
Per Jostein Matre1, Rita Kylling2, Pamela Waaler3, Hans Nordahl4, Kitty Dahl3 Background
1
Drammen Municipality, Drammen, Norway; 2Metanoia Mestring & Evidence suggests mental health interventions can be effectively de-
Utvikling, Borgen, Norway; 3RBUP Øst og Sør, Oslo, Norway; 4Norwegian
livered via task-sharing in low-resource settings [1]. However, re-
University of Science and Technology (NTNU), Trondheim, Norway
search focused on how to embed evidence-based treatments in
Correspondence: Per Jostein Matre ([email protected]) government-funded systems to enable population-level scale-up and
Implementation Science 2020, 15(Suppl 2):A148 sustainment is limited [2,3].
Background
We examined implementation policies and practices (IPPs) associated
Seventy-six present of youth living in child welfare institutions in with facilitating group-based trauma-focused cognitive behavioral
Norway meet criteria for 1 or more psychiatric disorders [1]. This high therapy (TF-CBT) for children and adolescents within the health and
prevalence of emotional distress presents a significant challenge for
education systems in Kenya. Eighteen teachers and 18 community
Norwegian child welfare services. In order to provide effective help,
health volunteers (CHVs) from each system (N = 36) participated in
institutional employees must be motivated, trained, and engaged. qualitative interviews after delivering 2 consecutive TF-CBT groups.
From 2011 to 2014, a systematic implementation called Module- Interviews examined several IPPs, including rewards and incentives.
Based Support (MBS) was adopted to increase the quality of institu-
Thematic coding was conducted by a team including the study’s
tional care through therapist training and supervision. The current
principal investigator. Interviews were double-coded and discussed
study examines which program factors contributed to therapist de- to consensus; a third coder was consulted when discordant.
velopment and job engagement. Results
Rewards and incentives were perceived differently between systems.
A quantitative cross-sectional design was utilized. The independent
Teachers highlighted rewards and incentives within their profession.
variable was job engagement, and High Performance Cycle (HPC) fac- Most teachers felt they were able to be more effective teachers due
tors [2] including: demands, performance, contingent rewards, conse- to participating in the intervention (72%), while only 44% of CHVs re-
quences, and job satisfaction were the dependent variables. Employees
ported improvement in their role due to participation. As salaried
and leaders in Norway’s Region North working day and/or evening
professionals, teachers did not receive compensation for participa-
shifts participated. In 2011, 320 employees and 12 leaders were re- tion; however, as volunteers, the CHVs received a stipend, which was
cruited (62% women, 38% men). The number of participants decreased endorsed by 61% as a reward/incentive. Further, CHVs—often con-
to 230 in 2013 due to organizational restructuring.
sidered the health system’s extension into the community—per-
Participants completed the Empowered Thinking Questionnaire (ETQ) 3
ceived additional rewards and incentives in relation to their
times between 2011 and 2014. Question topics involved: leader sup- communities that teachers did not. Nearly half of CHVs (44%) re-
port, goal orientation, self-efficacy, attachment to the organization, job ported receiving rewards and incentives from outside their job (e.g.,
engagement, job satisfaction, demands, and organizational practice [3].
gifts/acknowledgment from community members), whereas only 2
Results
teachers (11%) reported rewards/incentives from outside their profes-
A Structural Equation Modelling (SEM) analysis was conducted to test sion. CHVs reported benefit to their personal life slightly more fre-
the relationship between job engagement and the dependent vari- quently than teachers (56% v 44%).
ables. The analysis demonstrated a significant relationship between
Conclusions
demands and performance (b = .54), performance and contingent re-
Encouraging participation and sustaining task-sharing interventions
wards (b = .85), contingent rewards and job satisfaction (b = .94), requires understanding how rewards and incentives are perceived.
contingent rewards and job engagement (b = .58), and job satisfac- For professionals, like teachers, emphasizing rewards and incentives
tion and consequences (b = .88). However, significant relationships
related to their profession may encourage participation and enable
sustainment. For non-salaried volunteers, highlighting rewards and skills, and mentorship capabilities of researchers and policymakers re-
incentives from their communities might be more beneficial. In both garding evidence-based mental health treatment.
systems, counselors reported similar levels of personal benefit, like Trial Registration ClinicalTrials.gov NCT03711786
using skills to manage their own grief, suggesting participation goes
beyond profession and stipend to add value to counselors’ personal References
lives. 1. Patel V. Mental health in low- and middle-income countries. Br Med Bull.
Trial Registration ClinicalTrials.gov NCT01822366 2007;81–82(1):81–96. doi: 10.1093/bmb/ldm010.
2. Baxter AJ, Patton G, Scott KM, Degenhardt L, Whiteford HA. Global
References epidemiology of mental disorders: what are we missing? PloS One.
1. Patel V, Chowdhary N, Rahman A, Verdeli H. Improving access to 2013;8(6):e65514. doi: 10.1371/journal.pone.0065514.
psychological treatments: lessons from developing countries. Behav Res 3. Hanlon C, Luitel NP, Kathree T, Murhar V, Shrivasta S, Medhin G,
Ther. 2011;49(9):523-528. Ssebunnya J, Fekadu A, Shidhaye R, Petersen I, Jordans M, Kigozi F,
2. Fairall L, Zwarenstein M, Thornicroft G. The applicability of trials of Thornicroft G, Patel V, Tomlinson M, Lund C, Breuer E, De Silva M, Prince
complex mental health interventions. In: Thornicroft G, Patel V, editors. M. Challenges and opportunities for implementing integrated mental
Global Mental Health Trials. Oxford, UK: Oxford University Press; 2014. health care: a district level situation analysis from five low- and middle-
3. Betancourt TS, Chambers DA. Optimizing an era of global mental health income countries. PLoS One. 2014;9(2):e88437. doi: 10.1371/
implementation science. JAMA Psychiatry. 2016;73(2):99-100. journal.pone.0088437.
A151
A150 The Health Equity Implementation Framework: proposal and
A protocol for building mental health implementation capacity for preliminary study
Malawian and Tanzanian researchers and policymakers Eva Woodward1, Monica Matthieu1, Uchenna Uchendu2, Shari Rogal3,
Christopher Akiba1, Vivian Go1, Victor Mwapasa2, Mina JoAnn E. Kirchner4
Hosseinipour1, Brad Gaynes1, Alemayehu Amberbir3, Michael Udedi2, 1
VA Center for Mental Healthcare Outcomes Research, Little Rock, AR,
Brian Pence1 USA; 2Health Management Associates, Naples, FL, USA; 3VA Center for
1
University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; Health Equity Research & Promotion, Pittsburgh, PA, USA; 4VA Behavioral
2
University of Malawi College of Medicine, Blantyre, Malawi; 3Dignitas Health QUERI, Little Rock, AR, USA
International, Zomba, Malawi; 4Malawi Ministry of Health, Lilongwe, Correspondence: Eva Woodward ([email protected])
Malawi Implementation Science 2020, 15(Suppl 2):A151
Correspondence: Christopher Akiba ([email protected])
Researchers could benefit from methodological advancements to ad-
Background vance uptake of new treatments while also improving health equity.
Mental health disorders in low and middle-income countries (LMICs) A determinants framework for healthcare disparity implementation
account for nearly the same disease burden as HIV/AIDS [1,2]. While challenges is essential to understand an implementation problem
efficacious mental health treatments exist, access is severely limited and select implementation strategies.
[3]. This treatment gap is fueled by structural determinants rooted in Materials and Methods
a lack of research and policy capacity. We integrated and modified two conceptual frameworks—one from
Materials and Methods implementation science and one from healthcare disparities research
The goal of this abstract is to describe the capacity building proce- to develop the Health Equity Implementation Framework. We applied
dures and preliminary results of the sub-Saharan Africa Regional Part- the Health Equity Implementation Framework to a historical health
nership (SHARP) for Mental Health Capacity Building. SHARP 1) disparity challenge–Hepatitis C Virus (HCV) and its treatment among
strengthens implementation skills among Malawian and Tanzanian Black patients seeking care in the U.S. Department of Veterans Affairs
mental health researchers and policymakers to successfully apply re- (VA). A specific implementation assessment at the patient level was
search on evidence-based mental health programs into routine prac- needed to understand barriers to increasing uptake of HCV treat-
tice and 2) supports dialogue between researchers and policymakers ment, independent of cost. We conducted a preliminary study to as-
leading to efficient and sustainable scale-up of mental health ser- sess how feasible it was for researchers to use the Health Equity
vices. SHARP comprises five capacity building components for mental Implementation Framework. We applied the framework to design the
health researchers and policymakers including: 1) introductory and qualitative interview guide and interpret results. Using quantitative
advanced short courses focused on implementation science, data to screen potential participants, this preliminary study consisted
evidence-based mental health interventions, and grant writing; 2) a of semi-structured interviews with a purposively selected sample of
multifaceted dialogue platform; 3) an on-the-job training program; 4) Black, rural-dwelling, older adult VA patients (N=12), living with HCV,
annual pilot grants; and 5) mentorship courses. The impact of the from VA medical clinics in the Southern part of the United States.
program will be measured using dose, participant knowledge, partici- Results
pant satisfaction, and participant academic output. The Health Equity Implementation Framework was feasible for imple-
Results mentation researchers. Barriers and facilitators were identified at all
A group of 21 researchers and policymakers attended the introductory levels including the patient, provider (recipients), patient-provider
short courses (implementation science and evidence-based mental interaction (clinical encounter), characteristics of treatment
health interventions) in June 2018. Post-test knowledge scores increased (innovation), and healthcare system (inner and outer context). Some
by 87% and 15% respectively and received average user satisfaction rat- barriers reflected general implementation issues (e.g., poor care co-
ings of 89% and 85%. Pilot grants focused on implementation science ordination after testing positive for HCV). Other barriers were related
and mental health were awarded to 4 teams of one researcher and one to healthcare disparities and likely unique to racial minority patients
policymaker in August 2018. Also, SHARP partners delivered 3 journal (e.g., testimonials from Black peers about racial discrimination at VA).
clubs to 41 researchers and policymakers in 2018. We identified several facilitators, including patient enthusiasm to ob-
Conclusions tain treatment because of its high cure rates, and VA clinics that off-
Given the widespread lack of evidence-based mental health interven- set HCV stigma by protecting patient confidentiality.
tions brought to scale in LMICs, the experiences gained from the Conclusions
SHARP Capacity Building Program in Malawi and Tanzania will hold The Health Equity Implementation Framework showcases one way to
meaningful implications for a model of capacity building that could modify an implementation framework to better assess health equity
be replicated in other LMICs. If impactful, the SHARP Capacity Build- determinants as well. Researchers may be able to optimize the scien-
ing Program could be used to sustainably increase the knowledge, tific yield of research inquiries by identifying and addressing factors
that promote or impede implementation of novel treatments while Sycamores Child and Family Services (HSCFS) is one of the largest
also improving health equity. nonprofit community mental health and child welfare agencies in
Los Angeles County serving thousands of youth and families. Within
A152 this context the current study describes a two-phase pilot project for
Mental health disparities in CBT implementation evaluating the mechanisms underlying implementation of a perform-
Suh Jung Park, Hannah Listerud, Perrin Fugo, Emily Becker-Haimes, Rinad ance management ‘dashboard,’ which allows key metrics to be
S. Beidas tracked at multiple organizational levels (i.e., leadership to individual
University of Pennsylvania, Philadelphia, PA, USA clinicians).
Correspondence: Suh Jung Park ([email protected]) Materials and Methods
Implementation Science 2020, 15(Suppl 2):A152 Participants in the first phase of the pilot included 11 staff and the sec-
ond phase included 28 staff across two sites at HSCFS. In the first phase
Background of the pilot, the dashboards were distributed weekly and created manu-
The broader health care literature shows substantial racial and ethnic ally in Excel by agency staff. The second phase focused on identifying
disparities in the implementation of evidence-based practices (EBPs) and implementing a technology tool for developing and distribution the
[1], but there is a paucity of literature examining disparities in mental dashboards using Microsoft’s Power BI, where data updated daily. To
health care implementation. Cognitive-behavioral therapy (CBT) is a examine mechanisms that may impact implementation, Perceptions of
gold-standard EBP for multiple youth mental health conditions yet is Adopting an IT Innovation [3] and Readiness for Organizational Change
variably implemented in community mental health settings [2]. To [4] were administered before and after each phase.
date, little work has examined whether variability in CBT implemen- Results
tation differs as a function of client race and ethnicity; such findings Results of the implementation measures showed that participants in
would suggest the possibility of mental health disparities to target in the second phase had positive attitudes toward adoption (M=5.3,
future implementation research. SD=.83). They also indicated high leadership support for implementing
Materials and Methods the dashboard (M=5.2, SD=1.1). Based on the changes implemented in
We examined the relationship between therapists’ fidelity to CBT and the second phase (automating the dashboard; changing training strat-
clients’ race and ethnicity in community mental health settings using egies), participants reported positive attitudes particularly regarding
data drawn from a larger study. Therapists (N=72, median age=34 conveying the usefulness of the tool (M=5.7, SD=.86). Despite the posi-
years (IQR 29-44), 74% female, 69% white non-Hispanic/Latinx) tive attitudes utilization of the tool decreased in the second phase.
audio-recorded CBT therapy sessions for three unique clients. Ses- Conclusions
sions were coded for the extent to which therapists delivered six CBT Understanding which mechanisms underlie technology innovation
interventions (psychoeducation, cognitive education, cognitive distor- adoption is critical for identifying how to aid stakeholders ‘getting
tion, functional analysis of behavior, relaxation strategies, and coping ready’ for practices that may be different from current behavior. Des-
skills; other CBT interventions were excluded due to low use rates pite research demonstrating the impact of attitudes on clinician be-
across the sample) using the Therapy Process Observational Coding havior [5] positive perceptions of compatibility may not be enough
System for Child Psychotherapy-Revised Strategies Scale. We classi- to influence adoption at all levels. For instance, as this implementa-
fied clients into two groups: non-Hispanic/Latinx whites (N=53) and tion launches agency wide, it will be important to address de-
racial/ethnic minority (N=144) and examined whether therapist CBT implementation of other tools. Limitations to the current project in-
scores differed as a function of client racial/ethnic minority status. clude a limited sample size and preliminary nature of the project.
Results
Therapists used more relaxation strategies when treating racial/eth- References
nic minority clients than when treating non-Hispanic/Latinx white cli- 1. Carlier IVE, Meuldijk D, Van Vliet IM, Van Fenema E, Van der Wee NJA, Zitman
ents (t=-2.847, p=.005). No other differences were observed (all FG. Routine outcome monitoring and feedback on physical or mental health
ps>.05). status: evidence and theory. J Eval Clin Pract. 2012;18:104-110.
Conclusions 2. Bickman L, Kelley SD, Breda C, de Andrade AR, Riemer M. Effects of
Results did not suggest that therapists in community settings used routine feedback to clinicians on mental health outcomes of youth:
CBT differentially based on clients’ race and ethnicity. Study limita- results of a randomized trial. Psychiatr Serv. 2011;62:1423-1429.
tions include an overall restricted range of CBT intervention use, re- 3. Moore GC, Benbasat I. Development of an instrument to measure the
gardless of client racial/ethnic minority status. Implications for CBT perceptions of adopting an information technology innovation. Inf Syst
implementation will be discussed. Res. 1991;2(3):192-222.
4. Holt D, Armenakis A, Field H, Harris H. Readiness for organizational
References change: The systematic development of a scale. J Appl Behav Sci.
1. Roth A, Fonagy P. What works for whom? A critical review of 2007;43(2):232-255.
psychotherapy research. London, UK: Guilford; 1996. 5. Okamura KH. Therapists’ knowledge of evidence-based practice: Differen-
2. Hoagwood K, Burns BJ, Kiser L, Ringeisen H, Schoenwald SK. Evidence- tial definitions, measurement, and influence on self-reported practice
based practice in child and adolescent mental health services. Psychiatr (Order No. AAI10587357). 2018. https://search-proquest-com.weblib.li-
Serv. 2001;52(9),1179-1189. doi:10.1176/appi.ps.52.9.1179 b.umt.edu: 2443/docview/1951458808?accountid=14593
A153 A154
Putting it at their fingertips: mechanisms for increasing adoption Implementing internet-based CBT as part of a digital stepped care
of a technology-based performance management tool in a service in Australian general practice
community mental health organization Isabel Zbukvic1, Elizabeth Hanley1, Fiona Shand1, Helen
Cameo F. Stanick, Janine Quintero, Amanda Gentz, Gina Perez, Debbie Christensen1, Nicole Cockayne1, Christiaan Viis2, Josephine Anderson1
1
Manners Black Dog Institute, Randwick, Australia; 2Vrije Universiteit Amsterdam,
Hathaway-Sycamores Child and Family Services, Los Angeles, CA, USA Amsterdam, Netherlands
Correspondence: Cameo F. Stanick ([email protected]) Correspondence: Isabel Zbukvic ([email protected])
Research has demonstrated that simply giving providers progress iCBT is an effective resource for GPs, who are often the first contact
data on their clients produces an effect size for clinically significant for people experiencing mental ill-health and psychological distress
change [1-2]. However, the adoption of such innovations may be [1]. However, use of iCBT in primary care is not always routine. This
dependent on specific implementation constructs. Hathaway- study aims to improve GP and patient engagement with the
evidence-based iCBT program myCompass [2], through tailored im- improving the provision of healthcare services and patient outcomes.
plementation of Black Dog Institute’s digital stepped care service The Lean/Six Sigma Rapid Process Improvement Workshop (RPIW) and
StepCare [3]. This study forms part of the international implementa- related playbook are commonly used in VA quality improvement ef-
tion research project, ImpleMentAll [4]. forts. We used this approach as an implementation strategy to prepare
Materials and Methods adopters of eScreening in VA facilities.
StepCare is a digital mental health service designed for general prac- Materials and Methods
tice. Using a smart tablet, patients screen for symptoms of depres- We adapted a Lean Six Sigma RPIW [2] to develop an implementa-
sion, anxiety and alcohol misuse in the waiting room. Treatment tion strategy. Our multicomponent implementation strategy con-
recommendations based on symptom severity are sent from the tab- sisted of a modified RPIW, a playbook (roadmap for programs to
let to the GP in real-time; patients with mild symptoms are recom- implement eScreening, including how to conduct their own RPIW),
mended the self-guided iCBT program myCompass. Implementation- internal champions, and external facilitation through bi-monthly calls,
as-usual for the StepCare Service uses a train-the-trainer model, with a site visit, and technical assistance. We conducted a feasibility study
collaboration between the PHN and Black Dog Institute. Implemen- in 2 VHA clinics to evaluate the impact of RPIW on implementation.
ters in IT, practice support, research, and project management oper- We gathered qualitative data from site visits and consultation calls,
ate within an integrated model of knowledge translation to roll-out and conducted post implementation interviews. Two of our team
the service using a phased approach. members independently reviewed qualitative data using a rapid ana-
Results lytic approach to identify challenges and solutions.
Data collection for this study is currently underway. Following a Results
stepped-wedge trial design, a tailored implementation intervention The implementation strategy was used with slight variation across
(ItFits-toolkit) will be tested across 12 organizations in 9 countries the two sites. One site conducted a comprehensive RPIW (Site 1) and
over a period of 27 months. Longitudinal repeated-measures design the other relied on the other components of the implementation
will allow comparison of a baseline period of implementation-as- strategy (Site 2). Data from the pilot revealed three types of imple-
usual versus tailored implementation. Monthly implementation mentation challenges that occurred at both sites: technology-related,
reporting captures activities commenced/stopped and barriers over system level, and educational. Workflow and staffing resources were
time. Normalisation of GP use of myCompass as part of the StepCare challenges only at Site 2. All four types of implementation barriers
service will be measured quarterly using the NoMAD and ORIC. Out- were resolved using the external facilitator and the playbook.
come measures will also include patient uptake of myCompass via Conclusions
StepCare as well as implementation costs. Implementer experience Findings suggest that the use of the modified RPIW may have solved
using the ItFits-toolkit will also be assessed at each site, along with a workflow/staffing issues more efficiently than locally identified strat-
process evaluation of the ImpleMentAll trial. egies. The modified RPIW to implement new programs in VA health
Conclusions care systems shows promise. External facilitation helped overcome
The StepCare Service uses a technology solution to overcome known challenges with or without the RPIW. Our findings support prior re-
barriers to GP and patient engagement with online mental health search highlighting the importance of considering multiple change
care. This study will provide invaluable evidence on the effectiveness mechanisms of implementation strategies in mental health services
of tailored implementation strategies aimed at further improving en- [3].
gagement with iCBT in primary care.
References
References 1. Pittman JOE, Floto E, Lindamer L, Baker DG, Lohr JB, Afari N. VA
1. Parslow RA, Lewis V, Marsh B. The general practitioner’s role in providing escreening program: technology to improve care for post-9/11 veterans.
mental health services to Australians, 1997 and 2007: findings from the Psychol Serv. 2017; 14(1):23-33.
national surveys of mental health and wellbeing. Med J Aust. 2.Koning
2011;195(4):205-209. H, Verver JPS, Heuvel J, Bisgaard S, Does RJMM. Lean Six Sigma in healthcare.
2. Proudfoot J, Clarke J, Birch MR, Whitton AE, Parker G, Manicavasagar V, J Healthc Qual. 2006;28(2):4–11.
Harrison V, Christensen H, Hadzi-Pavlovic D. Impact of a mobile phone 3.Williams
and web program on symptom and functional outcomes for people with N. Multilevel mechanisms of implementation strategies in mental health:
mild-to-moderate depression, anxiety and stress: a randomised controlled integrating theory, research, and practice. Adm Policy Ment Health.
trial. BMC Psychiatry. 2013;13:312. 2015;43(5):783-796.
3. Institute BD. StepCare Service: A digital mental health screening tool for
patients in general practices. 2018. https://www.blackdoginstitute.org.au/
clinical-resources/health-professional-resources/stepcare-service. A156
4. ImpleMentAll. 2017. http://www.implementall.eu/. Accessed 27 Feb 2019. Implementation of an automated text messaging system for
patient self-management in the Department of Veterans Affairs: a
qualitative study
A155 Vera Yakovchenko, Timothy Hogan, Lorilei Richardson, Beth Ann
Development of a multicomponent implementation strategy for Petrakis, Christopher Gillespie, Derek Bolivar, D. Keith McInnes
eScreening Department of Veterans Affairs, Bedford, MA, USA
James Pittman1,2,3, Borsika Rabin1,3, Niloofar Afari1,2,3, Elizabeth Correspondence: Vera Yakovchenko ([email protected])
Floto4, Erin Almklov1, Laurie Lindamer1 Implementation Science 2020, 15(Suppl 2):A156
1
VA Center of Excellence for Stress and Mental Health, San Diego, CA,
USA; 2VA San Diego Healthcare System, San Diego, CA, USA; 3University Background
of California, San Diego, San Diego, CA, USA; 4VA Roseburg Health Care The Department of Veterans Affairs (VA) is currently deploying an au-
System, Roseburg, OR, USA tomated texting system (ATS) to support patient self-management.
Correspondence: James Pittman ([email protected]) Guided by the Non-adoption, Abandonment, Scale-Up, Spread, and
Implementation Science 2020, 15(Suppl 2):A155 Sustainability Framework (NASSS) which is intended to support the
evaluation of novel technologies, we conducted a qualitative study
Background to examine barriers and facilitators to national rollout of the ATS.
eScreening is a VA mobile health technology that provides customized Materials and Methods
and automated self-report health screening via iPad, clinical alerts, pa- Semi-structured interviews were conducted with 33 providers and 38
tient feedback and medical record integration [1]. eScreening supports patients at formative and summative stages of ATS implementation.
early identification of health problems and measurement-based care Interviews explored the roles of site personnel in ATS implementa-
initiatives. We set out to evaluate an implementation strategy for this tion, processes for enrolling patients, and ATS user experiences. Inter-
technology-based, self-screening tool that has shown promise for views were recorded and transcribed verbatim. Data were analyzed
via qualitative content analysis using emergent coding and a priori the use of Visual Abstract tweets and increased dissemination on so-
codes based on the NASSS framework. cial media. These findings may provide further evidence that Twitter
Results is an effective platform for research dissemination and highlight the
We identified themes across NASSS domains: 1) Condition: percep- importance of social media for suicide prevention researchers and
tions of patient appropriateness for the ATS were guided by texting other stakeholders to communicate findings. Future efforts will be
experience and health complexity rather than potential benefit; 2) discussed to implement Visual Abstracts at scale and refine processes
Technology: for providers, although the ATS resides outside the elec- to maximize engagement.
tronic health record, use was generally not considered laborious; 3)
Value Proposition: patient-driven demand for the ATS was limited; 4) References
Adopters: providers recommended more efficient ATS enrollment 1. Ibrahim AM. A primer on how to create a visual abstract. 2016.
processes to reduce workload; 5) Organization: providers did not www.SurgeryRedesign.com/resources.
have observable results from the ATS early in the implementation 2. Ibrahim A, Lillemoe K, Klingensmith M, Dimick J. Visual abstracts to
phase, noting that such evidence of patient progress/use could en- disseminate research on social media. Ann Surg. 2017;266(6):e46-e48. doi:
hance uptake among other providers; 6) Wider System: despite being 10.1097/SLA.0000000000002277
a national program, autonomy at the local level yielded varied expe- 3. Lindquist LA, Ramirez-Zohfeld V. Visual Abstracts to Disseminate Geriatrics
riences with the ATS; and 7) Embedding and Adaptation Over Time: Research Through Social Media. J Am Geriatr Soc. 2019. 7(6):1128-1131.
once using the ATS, providers recognized potential for use with other doi:10.1111/jgs.15853
conditions.
Conclusions
This is among the first studies to explore implementation of VA’s ATS A158
and through the lens of the NASSS framework. The NASSS framework Using web- and mobile technology to track implementation of
highlighted how the system can be better embedded into current evidence-based mental health treatments in schools
practices, which patients might benefit most from its functionality, Elizabeth Koschmann, Emily Berregaard, Shawna Smith, Seoyoun
and which aspects of ATS messages are potentially most relevant to Choi, Amy Rusch, John Hess
self-management. Mobile phone SMS texting is rapidly becoming an University of Michigan, Ann Arbor, MI, USA
accepted means of asynchronous communication between health- Correspondence: Elizabeth Koschmann ([email protected])
care systems and patients. Our findings reveal that VA’s ATS has po- Implementation Science 2020, 15(Suppl 2):A158
tential to expand the reach of VA care; however, providers require
additional support to adopt, implement, and sustain ATS use. Background
Forty percent of youth experience mental illness [1], yet access to
evidence-based treatments (EBTs) is limited by a shortage of trained
A157 providers, poor treatment fidelity, and low consumer help-seeking
Beyond journals – using visual abstracts to promote wider knowledge [2-3]. School-based delivery of EBTs can improve access
research dissemination [4] and research demonstrates that EBTs can be delivered effectively
Adam Hoffberg1, Joe Huggins1, Audrey Cobb1, Jeri Forster1,2, Nazanin by school-based mental health professionals (SMHPs) [5]. Implemen-
Bahraini1,2 tation strategies, such as consultation, coaching, and facilitation, pro-
1
Rocky Mountain MIRECC, Denver, CO, USA; 2University of Colorado, mote adoption and sustainment of EBTs in schools [6-8], however,
Boulder, CO, USA evaluation of implementation interventions requires collection of ap-
Correspondence: Adam Hoffberg ([email protected]) propriate implementation metrics [9] that is hindered in schools by
Implementation Science 2020, 15(Suppl 2):A157 absence of a unified, acceptable reporting system.
Background TRAILS is a statewide implementation model designed to increase
Many academic and journal organizations disseminate research via SMHP delivery of CBT, and incorporates didactic instruction, technical
social media to increase accessibility and reach a wider audience. support, and in-person coaching. To enable program evaluation, TRAI
With the widespread utilization of Twitter, more research is needed LS developed a web application, the TRAILS Dashboard, which allows
to study the extent to which social media strategies influence out- SMHPs to easily record weekly delivery of treatment components,
comes on awareness and readership of journal publications. “Visual self-reported fidelity, and track student clinical outcomes. Self-report
Abstracts” have been adopted by some organizations as a novel ap- data are cross-validated with standardized assessments and observer
proach to increase engagement with academic content. Visual Ab- (Coach) ratings to inform TRAILS implementation efforts.
stracts are a visual representation of key methods and findings Results
found in a traditional written publication [1]. This study will help or- Development of the TRAILS Dashboard included concept design, wire-
ganizations understand the potential impact of adopting Visual Ab- framing, user testing, and build. Currently, the Dashboard is being utilized
stracts into their social media dissemination efforts. Potential pitfalls within a randomized implementation-effectiveness trial, ASIC, evaluating
will also be discussed. implementation strategies. All collection of implementation (SMHP-level)
Materials and Methods and clinical effectiveness (student-level) data occurs through the Dash-
A prospective, case-control crossover design was utilized to randomize board, as does management of student suicide risk. A coach dashboard
n=50 journal publications comparing Twitter posts with a Visual Ab- documents delivery of coaching elements as well as observational ratings
stract to those with simple screen grab of the PubMed abstract. We of SMHP treatment fidelity. 169 SMHPs are actively using the Dashboard,
used native Twitter Analytics to track the outcomes of impressions, recording their weekly CBT delivery since November 2018; and have
retweets, total engagements, and link clicks about 28 days post-Tweet, identified 1,347 student participants for clinical outcomes data collection.
and Altmetric It to track additional alternative metric outcomes. Response rates for weekly data are consistently above 80% independent
Results of implementation strategy treatment arm.
As of this submission, n=47 out of n=50 articles have been random- Conclusions
ized, with complete follow-up data available for n=33 publications. The TRAILS Dashboard presents a novel solution to a common prob-
Preliminary analyses indicate that overall, Visual Abstract tweets on lem in non-clinical implementation research—the lack of a unified
average have 432 more impressions, 2 additional retweets, 1 add- system for tracking implementation outcomes. Deployment within
itional link click, 5 additional total engagements, and increase the the ASIC study allows testing of basic utility and ease of use, but on-
Altmetric score by 3 compared with Text Tweets. Full results from going initiatives are informing future functions to further tailor TRAI
the study will be analyzed and presented. LS implementation support, namely dynamic training targeting SMPH
Conclusions skill deficiencies, “nudges” to increase treatment frequency or fidelity,
Conclusions are pending, but it is expected that in line with results and prompts recommending specific treatment components for stu-
from prior studies [2,3] we will find a significant association between dents based on clinical data.
References for fuel use of household members and their quality of life. To assess
1. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, Benjet the implementation of facilitators and challenges, the planning team
C, Georgiades K, Swendsen J. Lifetime prevalence of mental disorders in conducted focus groups of key stakeholders and community leaders
U.S. adolescents: results from the National Comorbidity Survey to plan the intervention specifications. Data were analyzed qualita-
Replication—Adolescent Supplement (NCS-A). J Am Acad Child Adolesc tively using NVivo.
Psychiatry. 2010;49(10):980-9. doi: 10.1016/ j.jaac.2010.05.017. Results
2. Beidas RS, Kendall PC. Dissemination and implementation of evidence- The baseline survey results showed that 211 (59.8%) were currently
based practices in child and adolescent mental health. New York: Oxford using three-stone fire and 219 (79.0%) were using wood for their pri-
University Press; 2014 mary cooking fuel. 343 (97.2%) participants used charcoal as their al-
3. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness- ternative option for cooking fuel. 226 (64.0%) of the participants
implementation hybrid designs: combining elements of clinical effective- preferred a cookstove design with less smoke. The qualitative ana-
ness and implementation research to enhance public health impact. lysis revealed that enhancing cultural sensitivity and cultural rele-
Med Care. 2012;50(3):217-226. vance for adoption, implementation, and maintenance are relevant
4. Owens JS, Lyon AR, Brandt NE, Warner CM, Nadeem E, Spiel C, Wagner to effectiveness. Perceptions of complexibility can become a poten-
M. Implementation science in school mental health: key constructs in a tial barrier in implementing ICS.
developing research agenda. Sch Ment Health. 2013;6(2):99-111. Conclusions
5. Masia Warner C, Colognori D, Brice C, Herzig K, Mufson L, Lynch C, Reiss This baseline study shows that the ICS project can potentially have
PT, Petkova E, Fox J, Moceri DC, Ryan J, Klein RG. Can school counselors an impact on the targeted households. Ensuring the adoption of op-
deliver cognitive-behavioral treatment for social anxiety effectively? A timal and appropriate technologies by conducting formative research
randomized controlled trial. J Child Psychol Psychiatry. 2016;57(11):1229- can lead to facilitation of the intervention and improvement in the
1238. doi:10.1111/jcpp.12550. quality of the overall project.
6. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK,
Mandell DS. Methods to improve the selection and tailoring of References
implementation strategies. J Behav Health Serv Res. 2017;44(2):177-194. 1. University of Washington Institute for Health Metrics and Evaluation.
doi: 10.1007/s11414-015-9475-6. Nepal. 2018. http://www. healthdata.org/nepal. Accessed 25 February
7. Kilbourne AM, Smith SN, Choi SY, Koschmann E, Liebrecht C, Rusch A, 2019.
Abelson JL, Eisenberg D, Himle JA, Fitzgerald K, Almirall D. Adaptive 2. World Health Organization. Household air pollution and health. 2018.
School-based Implementation of CBT (ASIC): clustered-SMART for build- Available at: https://www.who. int/news-room/fact-sheets/detail/
ing an optimized adaptive implementation intervention to improve up- household-air-pollution-and-health. Accessed 1 April 2019
take of mental health interventions in schools. Implement Sci. 3. Jeuland M, Pattanayak SK Bluffstone RA. The economics of household air
2018;13(1):119. doi: 10.1186/s13012-018-0808-8. pollution. Annu Rev Resour Economics. 2015;7:81-108. doi:10.1146/
8. Koschmann ES, Abelson JL, Kilbourne AM, Smith SN, Fitzgerald K, annurev-resource-100814-125048
Pasternak A. Implementing evidence-based mental health practices in
schools: feasibility of a coaching strategy. (under review).
9. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, A160
Griffey R, Hensley M. Outcomes for implementation research: conceptual Ontario neurotrauma foundation’s journey: from traditional funder
distinctions, measurement challenges, and research agenda. Adm Policy to impact driven funding
Ment Health. 2011;38(2):65-76. doi: 10.1007/s10488-010-0319-7. Helene Gagne, Judith Gargaro
Ontario Neurotrauma Foundation, Toronto, Canada
Correspondence: Helene Gagne ([email protected])
Formative research findings from the metallic Improved Cooking
Stove (ICS) installation project for improving respiratory health in Background
Manekharka, Nepal Increasingly it is important to align funding priorities with practice
Jayoung Park1, Prabin Raj Shakya1, Sugy Choi2, Jongho Heo3, Woong- and system realities. It is necessary to ground research and know-
Han Kim1 ledge generation activities in real-world settings and engage broad
1
Seoul National University College of Medicine, Seoul, South Korea; stakeholders with interest in the research results in program and sys-
2
Boston University, Boston, MA, USA; 3National Assembly Futures tem planning, delivery and implementation [1]. As a funder we still
Institute, Seoul, South Korea aim to identify knowledge gaps and share the results with key stake-
Correspondence: Jayoung Park ([email protected]) holders, but we recognize that knowledge translation/mobilization al-
Implementation Science 2020, 15(Suppl 2):A159 though necessary cannot be enough to bring evidence-based
findings into practice.
In Nepal, the second highest ranking risk factor for death and disabil- Our approach has moved to funding fewer projects addressing prac-
ity combined is indoor air pollution (IAP) [1]. IAP is a risk factor for tice gaps and priorities in healthcare with a view to implementing re-
numerous diseases including pneumonia, stroke, ischemic heart dis- sults and innovations, using an implementation science framework.
ease, chronic obstructive pulmonary disease (COPD), and lung cancer The funding approach has further evolved from funding only individ-
[2]. Improved Cooking Stove (ICS) is known to reduce the exposure ual projects to now funding collaborative and system-level initiatives.
to IAP [3], but its implementation can have its challenges based on Evidence exists that implementation processes rooted in implemen-
its context and setting. The JWLEE CGM and Dhulikhel Hospital col- tation science can be effective, but there are issues of sustainability if
laborated to conduct formative research prior to the implementation the implementation is not embedded at the system level [2]. It is ne-
of the project. This study presents the results of the baseline survey cessary to foster partnerships and involve broad stakeholders in all
for phase 1 and qualitative results. phases of implementation.
Materials and Methods The key pillars of our integrative approach are knowledge gener-
A pragmatic hybrid type 1 design with a step-wedged trial is used. ation, knowledge mobilisation and effective implementation ap-
The study is focused on ward 4 for Panchpokhari-Thanpalkot rural proaches. As a funder this approach works well as we can specify the
municipality, Sindupalchowk District in the north-eastern part of type of services that we provide and can be nimble to respond to
Nepal. Out of all 480 households, a total of 363 households and a real-world challenges. Sometimes it is not possible to apply the full
total of 663 household individuals were surveyed for the baseline spectrum of implementation science activities from beginning to
study and data were collected digitally through CommCare HQ. The end; action must occur where it is needed to increase capacity of
household-level questions included the main cook’s fuel use, cooking implementing organisations within the system they work in. All our
dynamics, and health symptoms. The individual-level questions asked efforts are designed with a view for scalability and sustainability [3].
Results References
Within our three streams of activity (injury prevention, acquired brain 1. Tversky A, Kahneman D. The framing of decisions and the psychology of
injury and spinal cord injury) we have amassed a wealth of know- choice. Science. 1981; 211(4481):453-458.
ledge around implementation at the local and system levels and 2. Terwiesch C, Ulrich K. Innovation tournaments: creating and selecting
have developed an implementation support service resource to co- exceptional opportunities. Boston, MA: Harvard Business Review Press;
ordinate our implementation efforts in an explicit and consistent 2009.
way. 3. Beidas, RS, Volpp, KG, Buttenheim, AN Marcus SC, Olfson M, Pellecchia M,
Conclusions Stewart RE, Williams NJ, Becker-Haimes EM, Candon M, Cidav Z, Fishman
Examples will be discussed that illustrate our systematic approach J, Lieberman A, Zentgraf K, Mandell D. Transforming mental health deliv-
across the three streams and the lessons learned in moving from ery through behavioral economics and implementation science: protocol
traditional funding to impact funding. for three exploratory projects. JMIR Res Prot. 2019;8(2): e12121.
References
1. Rycroft-Malone J, Burton CR, Bucknall T, Graham ID, Hutchinson AM, Sta- A162
cey D. Collaboration and co-production of knowledge in healthcare: op- Examining how different engagement procedures in facilitated
portunities and challenges. Int J Health Policy and Manag. 2016;5(4):221– interprofessional collaborative processes optimize type-2 diabetes
3. doi: 10.15171/ ijhpm.2016.08. prevention in routine primary care
2. Nilson P. Making sense of implementation theories, models and Alvaro Sanchez1, Gonzalo Grandes2, Susana Pablo2, Arturo García-
frameworks. Implement Sci. 2015;10(53). doi: 10.1186/s13012-015-0242-0. Álvarez2
1
3. Sustainable Improvement Team and the Horizons Team. Leading large- Osakidetza-Basque Health System, Bilbao, Spain; 2BioCruces Bizkaia
scale change: a practical guide. 2018. https://www.england.nhs.uk/wp- Health Research Institute, Osakidetza, Bizkaia, Spain
content/uploads/2017/09/practical-guide-large-scale-change-april-2018- Correspondence: Alvaro Sanchez ([email protected])
smll.pdf. Accessed 2018 March 2019. Implementation Science 2020, 15(Suppl 2):A162
Background
A161 Most efficient procedure to engage and guide healthcare profes-
New directions for implementation strategy design: applying sionals in collaborative processes that seek to optimize practice is un-
behavioral economic insights to design EBP implementation known [1]. The PREDIAPS project aims to assess the effectiveness
strategies in community mental health settings and feasibility of different engagement procedures to perform a facil-
Vivian Byeon1, Rebecca Stewart1, Rinad Beidas1, Briana Last1, Katelin itated interprofessional collaborative process to optimize type-2 dia-
Hoskins1, Nathaniel Williams2, Alison Buttenheim1 betes prevention in routine Primary Care [2].
1
University of Pennsylvania, Philadelphia, PA, USA; 2Boise State Materials and Methods
University, Boise, ID, USA Randomized cluster implementation trial conducted in nine PHC cen-
Correspondence: Vivian Byeon ([email protected]) ters from the Basque Health Service. All centers received training on
Implementation Science 2020, 15(Suppl 2):A161 effective healthy lifestyles promotion. Headed by a local leader and
an external facilitator, centers conducted a collaborative structured
Background process to adapt the intervention and its implementation to their
The field of behavioral economics provides ro bust explanations for specific context [3]. One of the groups was allocated to apply this
why individuals choose and behave as they do [1], yet little work has strategy globally, promoting the cooperation of all health profes-
applied these insights to the development of implementation strat- sionals from the beginning. The other performed it sequentially, cen-
egies that influence community mental health clinicians’ choices and tered first on nurses, who lately seek the pragmatic cooperation of
behavior with regard to evidence-based practice. In this study, a physicians. All patients without diabetes aged ≥30 years old with a
team of behavioral scientists and frontline mental health clinicians in known CVD risk factor and an abnormal glucose level (≥110-125 mg/
the city of Philadelphia used a systematic approach to the identifica- dl) who attended centers during the study period were eligible for
tion of behavioral barriers to EBP use in order to develop novel im- program inclusion. Main outcome measures focus on changes in T2D
plementation strategies for community mental health settings. This prevention practice indicators after 12 months.
poster describes our approach and results. Results
Materials and Methods Exposition rate of professionals to the implementation strategy ac-
We followed a four-step process to 1) define our target problem; 2) tions were similar in both groups but higher in nurses (86%) than in
map the relevant decisions and actions underlying the behavior; 3) physicians (75%). After 12 months, 2916 eligible at risk patients
brainstorm hypothesized behavioral barriers using previously- attended at least once to their family physician, of which 401 (13.8%)
collected contextual inquiry data (implementation strategy ideas that have been addressed by assessing their healthy lifestyles in both
therapists generated during an innovation tournament) [2-3], linked comparison groups. The proportion of attending patients at risk of
to specific behavioral science constructs; and 4) conduct rapid valid- T2D receiving a personalized prescription of a healthy lifestyle
ation of hypothesized barriers through expert consultation, literature change (N=214; 7.3%) was slightly higher in the Sequential (7.8%;
review, and a clinician focus group. range 5.5%-10.8%) than in the Global group (6.3%; range 5.8%-6.7%).
Results The proportion of patients receiving a lifestyle prescription from
Drawing on the crowdsourcing data from clinicians, the investigative those assessed is also higher in the Sequential than in the Global
team generated 156 hypotheses of behavioral barriers to EBP use. Two group (55% vs. 50%).
investigators then de-duplicated and synthesized hypotheses down to Conclusions
a list of 21. We are currently in the last stages of the rapid validation Preliminary results showed that the reach of the implanted interven-
process with community clinicians and will present the final hypotheses tion programs derived by the PREDIAPS implementation strategy is
to support implementation strategy design in our poster session. acceptable but slightly higher in the Sequential group. Center’s
Conclusions organizational context has determined implementation results (pro-
To our knowledge, this is the first study incorporating principles of fessional commitment, work overload, multiple corporative initiatives,
behavioral economics and participatory design to the development staff turn-over, etc.).
of implementation strategies. This systematic, rigorous, and innova-
tive process will allow us to develop candidate implementation strat- References
egies from the insights of clinicians. Results from this participatory 1. Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, Everitt H,
and behavioral process will drive the design of tailored implementa- Kennedy A, Qureshi N, Rogers A, Peacock R, Murray E. Achieving change
tion strategies that will explicitly address and overcome the identi- in primary care—causes of the evidence to practice gap: systematic
fied behavioral barriers. reviews of reviews. Implement Sci. 2016;11:40.
2. Sanchez A, Grandes G, Pablo S, Espinosa M, Torres A, García-Alvarez A; primary care based on the principles of academic detailing. J Contin
PREDIAPS Group. Engaging primary care professionals in collaborative Educ Health Prof. 2018; 38(4):269-275.
processes for optimising type 2 diabetes prevention practice: the PREDIA 3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
PS cluster randomised type II hybrid implementation trial. Implement Sci. MM, Proctor EK, Kirchner JE. A refined compilation of implementation
2018;13(1):94. strategies: results from the Expert Recommendations for Implementing
3. Grandes G, Sanchez A, Cortada JM, Pombo H, Martinez C, Balagué L, Change (ERIC) project. Implement Sci. 2015;10(1):21. doi: 10.1186/s13012-
Corrales MH, de la Peña E, Mugica J, Gorostiza E; PVS group. 015-0209-1.
Collaborative modeling of an implementation strategy: a case study to
integrate health promotion in primary and community care. BMC Res A164
Notes. 2017;10(1):699 Learning by doing: implementing the VA Cardiovascular Toolkit
and learning to fail better
Bevanne Bean-Mayberry1, Erin Finley2, Alison Hamilton1,3, Tannaz Moin1,3,
A163 Melissa Farmer1,3
An approach to align barriers and implementation strategies to 1
VA Greater Los Angeles, Los Angeles, CA, UA; 2STVHCS/UTHSA, San Antonio,
accelerate adoption of evidence-based practice: CVD risk calculator TX, USA; 3University of California, Los Angeles, Los Angeles, CA, USA
adoption in primary care Correspondence: Bevanne Bean-Mayberry (bevanne.bean-
Laura-Mae Baldwin1, Leah Tuzzio2, Allison Cole1, Erika Holden2, Jennifer [email protected])
Powell3, Michael Parchman2 Implementation Science 2020, 15(Suppl 2):A164
1
University of Washington, Seattle, WA, USA; 2Kaiser Permanente
Washington Health Research Institute, Seattle, WA, USA; 3Powell & Background
Associates, Ashville, NC, USA Cardiovascular (CV) disease is the main cause of death in American
Correspondence: Laura-Mae Baldwin ([email protected]) women and CV risk factors are often less controlled in women com-
Implementation Science 2020, 15(Suppl 2):A163 pared to men. To address CV risk among women Veterans, we devel-
oped a CV toolkit comprised of a patient screener, an electronic health
Background record template, and a health education group tailored for women to
Healthy Hearts Northwest (H2N), one of seven AHRQ-funded Eviden- target CV goal-setting. We describe how iterative cycles of adaptation
ceNOW cooperatives, is a pragmatic clinical trial to test different during implementation allows us to improve fit with local context, ad-
strategies for implementing evidence-based interventions to improve dress barriers, and increase engagement with CV toolkit components.
heart health in primary care practice [1]. One intervention was a vir- Materials and Methods
tual educational outreach visit (EOV) to increase use of cardiovascular Guided by Replicating Effective Programs (REP) enhanced with com-
disease (CVD) risk calculation to inform statin use for prevention of plexity theory, we are conducting an 18-month CV Toolkit study in
CVD [2]. Five physician educators conducted 30-minute EOVs in 44 four sites to increase engagement in CV risk reduction services. Dur-
H2N practices and elicited 13 barriers to implementing CVD risk ing implementation, we monitor staff/provider use of the template
calculation. and track patient engagement in VA services. Periodic reflections are
Materials and Methods used to review monthly progress and document multilevel stake-
To compare the implementation strategies that implementation sci- holder engagement and sense-making.
entists and primary care clinicians chose as most likely to overcome Results
barriers to implementing a CVD risk calculator in practice. Despite early and frequent engagement, there was limited utilization of
Modified nominal group exercise involving implementation scientists the patient screener and template over a one-year period at the first
and primary care clinicians, with synthesis of the exercise results Par- site, with staff and providers reporting difficulty remembering to use
ticipants: 5 implementation scientists and 26 clinicians from primary these tools. While providers made frequent referrals to health educa-
care clinics in the WWAMI region Practice and Research Network. tion group, participation was low, and remained low even after employ-
Every implementation scientist and clinician chose their top 5 imple- ing several strategies to increase participation, including audio-care
mentation strategies from a list of the 73 evidence-based, published calls and secure messaging. Implementation at later sites indi-
strategies from the Expert Recommendations for Implementing cated that scheduling patients for group classes increased partici-
Change (ERIC) study [3] for the 13 barriers. For each barrier, we ex- pation and clinical reminders helped care teams incorporate CV
amined the degree of agreement among ≥30% of clinicians, ≥30% of toolkit into routine workflow; these suggestions were then incor-
scientists, or ≥30% of both on chosen strategies. porated at the first site. Stakeholder (patient, provider and oper-
Results ational partners) feedback led us to develop a telephone
≥ 30% of clinicians and/or scientists chose 39/73 top implementation facilitated group to increase engagement.
strategies they felt were the most important to address barriers to Conclusions
CVD risk calculation. Implementation scientists chose 34 of these 39 The emphasis on nonlinear cycles of tailoring and adaptation in REP
top strategies; clinicians chose 21. Scientists and clinicians agreed on gave our team a structured process by which to proactively learn
the choice of 14 of the top implementation strategies. On average, a from early failures, engaging stakeholders consistently in identifying
total of 7 top implementation strategies were chosen by either scien- pragmatic solutions that aligned with routine workflow. These solu-
tists or clinicians for each barrier; however, scientists and clinicians tions became real-time modifications to the implementation process
agreed on only 1 of these top strategies. to mold tools to the local context, observe uptake and document
Conclusions outcomes.
Implementation scientists and clinicians generally choose different Trial Registration: ClinicalTrials.gov NCT02991534
implementation strategies to overcome barriers to implementing a
CVD risk calculator in practice. Collaboration between these stake-
holders could guide the choice of a broader range of strategies to A165
overcome barriers to evidence-based practice. Longitudinal assessment of Expert Recommendations for
Trial Registration: ClinicalTrials.gov NCT02839382 Implementing Change (ERIC) strategies in the uptake of evidence-
based practices for Hepatitis C treatment in the Veterans
References Administration
1. Parchman ML, Anderson ML, Dorr DA, Fagnan LJ, O’Meara ES, Tuzzio L, Vera Yakovchenko1, Rachel Gonzalez1, Angela Park1, Timothy
Penfold RB, Cook AJ, Hummel J, Conway C, Cholan R, Baldwin L-M. A ran- Morgan1, Maggie Chartier1, David Ross1, Matthew Chinman2, Shari Rogal1
1
domized trial of external practice support to improve cardiovascular risk Department of Veterans Affairs, Bedford, MA, United States; 2RAND
factors in primary care. Ann Fam Med. 2019;17(Supp 1):S40-S49. Corporation, Pittsburgh, PA, USA
2. Baldwin LM, Fischer MA, Powell J, Holden E, Tuzzio L. Fagnan LJ, Correspondence: Vera Yakovchenko ([email protected])
Hummel J, Parchman ML. Virtual educational outreach intervention in Implementation Science 2020, 15(Suppl 2):A165
Background vaccination services for adolescents. To date, no study we are aware

To increase access to evidence-based treatments for hepatitis C of has documented the experiences of implementing HPV vaccin-
(HCV), the Department of Veterans Affairs (VA) established a national ation programs in real-world pharmacy settings. We sought to docu-
collaborative composed of regional teams of providers, leaders, and ment challenges and opportunities of implementing pharmacy-
staff tasked with conducting local implementation strategies to in- located HPV vaccination services in five US states.
crease HCV treatment initiations. The aim of this longitudinal evalu- Materials and Methods
ation was to assess how site-level implementation strategies were We evaluated the success of the pilot projects by mapping reported
associated with HCV treatment initiation. results to key implementation science constructs: service penetration,
Materials and Methods acceptability, appropriateness, feasibility, fidelity, adoption, and sus-
A representative from each VA site (N=130) was asked in four con- tainability [4,5]. Pilot projects were planned in North Carolina (k=2
secutive fiscal years (FYs) to complete an online survey examining pharmacies), Michigan (k=10), Iowa (k=2), Kentucky (k=1), and Ore-
use of 73 implementation strategies organized into nine clusters as gon (no pharmacy recruited) with varying procedures and recruit-
described by the Expert Recommendations for Implementing Change ment strategies. Sites had open enrollment for a combined 12
(ERIC) study. The number of Veterans initiating treatment for HCV, or months.
“treatment starts”, at each site was captured using administrative Results
data. Descriptive, nonparametric, and multivariate analyses were con- Despite substantial efforts in these states, only 13 HPV vaccine doses
ducted on the respondents in FY15 (N=80), FY16 (N=105), FY17 (N= were administered to adolescents and three doses to age-eligible
109), and FY18 (N=88). young adults. We identified two major reasons for these underper-
Results forming results. First, poor outcomes on service penetration and ap-
Of 130 sites, 127 (98%) responded at least once and 54 (42%) propriateness pointed to engagement barriers: low parent demand
responded across all four years. A mean of 25±14 strategies were en- and engagement among pharmacy staff. Second, poor outcomes on
dorsed in FY15, 28±14, 26±15, and 35±26 in FY16, FY17, and FY18, feasibility, adoption, and sustainability appeared to result from ad-
respectively. While the number of strategies increased over time, the ministrative hurdles: lacking third party reimbursement (i.e., billing
correlation between number of strategies and HCV treatment de- commercial payers, participation in Vaccines for Children program)
creased over time. The most commonly endorsed strategies across and limited integration into primary care systems.
all years were: data warehousing techniques, tailoring strategies to Conclusions
deliver HCV care, and intervening with patients to promote uptake In summary, pilot projects in five states all struggled to administer
and adherence to HCV treatment. One strategy (“make efforts to HPV vaccines. Opportunities for making pharmacies a successful set-
identify early adopters to learn from their experiences”) was signifi- ting for adolescent HPV vaccination include expanding third party re-
cantly associated with treatment starts in all four years. In FY15, strat- imbursement to cover all vaccines administered by pharmacists,
egies were focused on developing interrelationships, FY16 focused increasing public awareness of pharmacists’ immunization training,
on using evaluative and iterative strategies, FY17 focused on training and improving care coordination with primary care providers.
and educating stakeholders, and FY18 focused on providing inter-
active assistance. The important strategies in each year were then References
mapped to Exploration, Preparation, Implementation, Sustainment 1. Walker TY, Elam-Evans LD, Yankey D, Markowitz LE, Williams CL, Mbaeyi
framework stages. SA, Fredua B, Stokley S. National, regional, state, and selected local area
Conclusions vaccination coverage among adolescents aged 13–17 years—United
This evaluation represents the first large-scale four-year assessment States, 2017. MMWR Morb Mortal Wkly Rep. 2018;67(33):909.
of implementation strategies nationwide. Surveying providers about 2. President’s Cancer Panel. HPV vaccination for cancer prevention:
ERIC strategies is a feasible way to understand the associations be- progress, opportunities, and a renewed call to action. a report to the
tween strategies and clinical outcomes over time. These results add President of the United States from the C=chair of the President’s Cancer
to our understanding of the implementation strategies used over Panel. Bethesda, MD. 2018.
time and across stages of planning, implementation, and 3. National Vaccine Advisory Committee. Recommendations to address low
sustainability. HPV vaccination coverage rates in the United States. June 9, 2015.
Trial Registration ClinicalTrials.gov NCT04178096 4. Gerke D, Lewis E, Prusaczyk B, Hanley C, Baumann A, Proctor E. 2017.
Eight toolkits related to Dissemination and Implementation.
Implementation Outcomes. https://sites.wustl.edu/wudandi/. Accessed 28
A166 Nov 2018.
Implementing pharmacy-located hpv vaccination: findings from 5. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A,
pilot projects in five U.S. states Griffey R, Hensley M. Outcomes for implementation research: conceptual
William Calo1, Parth Shah2, Melissa Gilkey3, Robin Vanderpool4, Sarah distinctions, measurement challenges, and research agenda. Adm Policy
Barden5, William Doucette6, Noel Brewer3 Ment Health. 2011;38(2):65-76.
1
Pennsylvania State University, University Park, PA, USA; 2Fred
Hutchinson Cancer Research Center, Seattle, WA, USA; 3University of
North Carolina, Chapel Hill, NC, USA; 4University of Kentucky, Lexington, A167
KY, USA; 5Michigan Pharmacists Association, Lansing, MI, USA; 6University Identifying facilitators and barriers for the implementation of a
of Iowa, Iowa City, IA, USA complex intervention for patients with metastatic lung cancer
Correspondence: William Calo ([email protected]) Anja Siegle1, Corinna Jung1, Nicole Deis2, Jasmin Bossert1, Katja
Implementation Science 2020, 15(Suppl 2):A166 Krug1, Michel Wensing1, Jana Jünger2, Michael Thomas1, Matthias
Villalobos1
1
Background Heidelberg University Hospital, Heidelberg, Germany; 2The German
Up-to-date human papillomavirus (HPV) vaccination in the US has in- National Institute for State Examinations in Medicine, Pharmacy and
creased since the vaccine’s introduction over a decade ago to 49% Psychotherapy, Mainz, Germany
of adolescents ages 13-17 in 2017 [1]. However, vaccination coverage Correspondence: Anja Siegle ([email protected])
remains far below the Healthy People 2020 goal of 80% for adoles- Implementation Science 2020, 15(Suppl 2):A167
cents ages 13-15 [1]. As a strategy to improve uptake, the President’s
Cancer Panel [2] and the National Vaccine Advisory Committee [3] Background
have recommended expanding HPV vaccine provision in pharmacies. The German National Cancer Plan and the American Society of Clin-
Pharmacies are promising alternative settings for HPV vaccination be- ical Oncology (ASCO) [1] recommend early integration of palliative
cause of their population reach, convenience, and existing infrastruc- care and advance care planning for patients with metastatic lung
ture for vaccine delivery. As a result, pilot projects conducted in five cancer. To address these recommendations, the Heidelberg Mile-
states aimed to demonstrate the utility of pharmacy-located HPV stone communication approach (MCA) has been developed [2]. MCA
is a complex intervention involving tandems of physicians and across Texas [1-3]. Via a mixed methods [4,5] approach we used a
nurses. It aims at providing coherent care along the disease trajec- formative evaluation process to adapt implementation strategies to
tory and integrates palliative care early. The MCA has been imple- local contexts and evaluated program outcomes and characterized
mented in a theory-led way [3] in the in- and out-patient processes influencing implementation in two local mental health au-
departments of a comprehensive cancer center (hospital setting). thorities serving 17 clinics.
While the degree of the implementation success is still under scru- Materials and Methods
tiny, the importance of identifying barriers and facilitators of the Varied data collection included pre and post-implementation leader,
MCA is already apparent. The aim of this study is to identify barriers provider, and staff surveys; and pre, mid, and post-implementation
and facilitators of the MCA implementation. provider, staff and consumer focus groups. During implementation,
Materials and Method data were collected via various logs (tobacco screenings, nicotine re-
A qualitative content analysis [4] of written minutes (n= 47) of implemen- placement therapy delivery) to monitor program content delivery.
tation meetings with nurses, doctors and hospital managers is con- Results
ducted. The data analysis comprises open coding, development of main All clinics adopted a 100% tobacco-free workplace policy, integrated
categories, identification of sub categories and application of these cat- tobacco screenings into routine practice, delivered evidence-based
egories. As a theoretical framework for implementation evaluation of the interventions, dispensed nicotine replacement therapies to con-
MCA, the Consolidated Framework for Implementation research (CFIR) [5] sumers and staff, and recorded significant increases in provider
is used. MAXQDA is used to organize the collected data. knowledge on how to address tobacco dependence. Pre, mid, and
Results post-implementation qualitative findings served to: 1) develop pro-
Preliminary results will be presented on aspects facilitating or hinder- gram strategies (educational tools, videos) and materials (brochures,
ing implementation in the following dimensions: characteristics of posters) adapted to local contexts and populations and address bar-
the intervention (e.g. adaption, fit), inner (e.g. communications, cli- riers; 2) adjust delivery systems of key components to enhance im-
mate, readiness) and outer (e.g. economic, political, social context) plementation; 3) understand reasons for success or failure to
settings, individuals involved (e.g. interaction between individuals, in- implement specific practices, respectively; and 4) reveal program in-
fluence of individual or organizational behavior) and implementation tegration into clinic culture, enhancing sustainability.
process (and sub-processes). Conclusions
Conclusions TTTF has proven successful in integrating tobacco cessation interven-
Knowing facilitators and barriers of the MCA supports future imple- tions into regular clinical practice to address tobacco use within be-
mentation processes in this hospital. Understanding this implementa- havioral health clinics. Mixing methods involved program adopters
tion process helps to identify determinants for successful and recipients as collaborators who directly impacted implementa-
implementation processes of the MCA in other organizations. tion by shaping the intervention to their individual context and
needs. Collaboration of such key stakeholders was vital to increasing
References program fit, ownership, adoption and sustainability; closing the gap
1. Peppercorn JM, Smith TJ, Helft PR, Debono DJ, Berry SR, Wollins DS, between research and practice. These findings contribute to the de-
Hayes DM, Von Roenn JH, Schnipper LE; American Society of Clinical velopment of flexible strategies and tailored interventions responsive
Oncology. American Society of Clinical Oncology statement: toward to real-world conditions in diverse settings which better address im-
individualized care for patients with advanced cancer. J Clin Oncol. plementation barriers thus enhancing the effectiveness and sustain-
2011;29(6):755-760. ability of a tobacco-free workplace program.
2. Siegle A, Villalobos M, Bossert J, Krug K, Hagelskamp L, Krisam J, Handtke
V, Deis N, Jünger J, Wensing M, Thomas M. The Heidelberg Milestones References
Communication Approach (MCA) for patients with prognosis <12 1. Correa-Fernández V, Wilson WT, Kyburz B, O’Connor DP, Stacey T, Wil-
months: protocol for a mixed-methods study including a randomized liams T, Lam CY, Reitzel LR. Evaluation of the Taking Texas Tobacco Free
controlled trial. Trials. 2018;19(1):1-13. Workplace Program within behavioral health centers. Transl Behav Med.
3. Grol R, Wensing M, Eccles M, Davis D. Improving patient care: the 2019;9(2):319-327
implementation of change in health care. 2nd ed. Chichester: John Wiley 2. Correa-Fernández V, Wilson WT, Shedrick DA, Kyburz B, L Samaha H, Sta-
& Sons, Ltd; 2013. cey T, Williams T, Lam CY, Reitzel LR. Implementation of a tobacco-free
4. Kuckartz U. Qualitative text analysis: a guide to methods, practice and workplace program at a local mental health authority. Transl Behav Med.
using software. London: Sage Publications Ltd; 2014. 2017;7(2):204-211.
5. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. 3. Samaha HL, Correa-Fernández V, Lam C, Wilson WT, Kyburz B, Stacey T,
Fostering implementation of health services research findings into Williams T, Reitzel LR. Addressing tobacco use among consumers and
practice: a consolidated framework for advancing implementation staff at behavioral health treatment facilities through comprehensive
science. Implement Sci. 2009; 4:50. workplace programming. Health Promot Prac. 2017;18(4):561-570.
4. Teddlie C, Tashakkori A. Major issues and controversies in the use of
mixed methods in the social and behavioral sciences. Handbook of
A168 mixed methods in social & behavioral research. Thousand Oaks, CA:
Using mixed methods to adapt and evaluate the implementation SAGE; 2003. p. 3-50.
of a comprehensive tobacco-free workplace program within 5. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk
behavioral health care facilities in Texas J. Mixed method designs in implementation research. Adm Policy Ment
Isabel Martinez Leal1, Kathy Le1, Daniel O’Connor1, Bryce Health. 2011;38(1):44-53.
Kyburz2, Virmarie Corrrea-Fernández1, Teresa Williams2, Lorraine Reitzel1
1
University of Houston, Houston, TX, USA; 2Integral Care, Austin, TX, USA
Correspondence: Isabel Martinez Leal ([email protected]) A169
Implementation Science 2020, 15(Suppl 2):A168 Building practitioner competency in implementation science to
drive comprehensive cancer control planning
Background Correspondence: Margaret Farrell ([email protected])
Despite the highest rates of tobacco use and tobacco-related mor- National Cancer Institute, Bethesda, MD, USA
bidity and mortality, smokers with behavioral health disorders rarely Implementation Science 2020, 15(Suppl 2):A169
receive tobacco dependence treatment within behavioral health care
settings. Taking Texas Tobacco Free (TTTF) has successfully targeted Background
this disparity by delivering a multi-component, tobacco-free work- A greater understanding and uptake of implementation science
place program providing policy implementation and enforcement, frameworks and measures can help both cancer control practitioners
education, provider training in tobacco screenings and treatments, and researchers leverage crucial insights into how to best deliver
and nicotine replacement therapies to behavioral health clinics research-based initiatives in the complex communities where they
are crucially needed. In April, the National Cancer Institute (NCI) re- providers (PCP), 12 pharmacists and pharmacy technicians, and 12 pa-
leased Implementation Science at a Glance, a workbook to introduce tients (n=36). PCP and pharmacy staff interviews are focused on care co-
practitioners and policymakers to the building blocks of implementa- ordination and follow-up processes and procedures, whereas patient
tion science. This resource is a natural extension of our experiences interviews are focused on the acceptability and relative advantage of re-
funding [1] and training researchers in implementation science as ceiving FIT kits from pharmacies. Interviews are audio-recorded, tran-
well as perspectives NCI gained through this work [2]. scribed and independently coded by two team members using a
Materials and Methods directed content analysis approach [8, 9] PRISM [10] and Diffusion of In-
A preliminary draft of the resource was reviewed by fifty-eight cancer novations Theory guide analysis and organization of themes.
control researchers and practitioners for clarity and concept. The final Results
version reflected advances both in our understanding of implemen- Pharmacies are well-positioned to increase access to preventive health
tation science and how to communicate it to support and inform the services such as colorectal cancer screening. Patients, pharmacists, and
work of cancer control practitioners. primary care providers have voiced support for an extended role for
Results pharmacists in delivering FIT kits for colorectal cancer screening.
Case studies illustrate implementation science in practice, provide Conclusions
lessons learned in the field, and brief practitioners about the compo- Results from this study can be used to elucidate key care coordin-
nents of IS including evidence-based interventions, fidelity, adapta- ation and follow-up issues for primary care providers and pharmacy
tions, stakeholder engagements, theories, models, and frameworks, staff and to identify implementation strategies [11] needed to target
strategies, evaluation, and sustainability. identified barriers (e.g., training pharmacists) to test intervention im-
Conclusions plementation and effectiveness.
This presentation will outline how Implementation Science at a
Glance illustrates implementation science frameworks, models and
References
measures to help drive community and organizational transformation
1. Siegel RL, Sahar L, Robbins A, Jemal A. Where can colorectal cancer
and, in turn, develop broader disparities-reducing implementation
screening interventions have the most impact? Cancer Epidemiol
strategies. By providing insights supporting greater uptake of Imple-
Biomarkers Prev. 2015;24(8):1151-1156.
mentation science research designs, the resource offers rigorous
2. Cole AM, Jackson JE, Doescher M. Urban-rural disparities in colorectal
methods that could accelerate the pace at which equity is achieved
cancer screening: cross-sectional analysis of 1998-2005 data from the
in real-world practice.
Centers for Disease Control’s Behavioral Risk Factor Surveillance Study.
Cancer Med. 2012; 1(3):350-6.
References
3. Blake KD, Moss JL, Gaysynsky A, Srinivasan S, Croyle RT. Making the case
1. Neta G, Sanchez MA, Chambers DA, Phillips SM, Leyva B, Cynkin L, Farrell
for investment in rural cancer control: An analysis of rural cancer
MM, Heurtin-Roberts S, Vinson C. Implementation science in cancer pre-
incidence, mortality, and funding trends. Cancer Epidemiol Biomarkers
vention and control: a decade of grant funding by the National Cancer
Prev. 2017;26(7):992-997.
Institute and future directions. Implement Sci. 2015;10(1):4
4. Kuo TM, Meyer AM, Baggett CD, Olshan AF. Examining determinants of
2. National Cancer Institute. Implementation Science at a glance: a guide
geographic variation in colorectal cancer mortality in North Carolina: A
for cancer control practitioners. [Bethesda, MD]: US Department of Health
spatial analysis approach. Cancer Epidemiol. 2019;59:8-14.
and Human Services, National Institutes of Health, National Cancer
5. Pignone M, Rich M, Teutsch SM, Berg AO, Lohr KN. Screening for colorectal
Institute, 2019. NIH Publication Number 19-CA-8055.
cancer in adults at average risk: a summary of the evidence for the U.S.
Preventive Services Task Force. Ann Intern Med. 2002;137(2):132-141.
A170 6. Knudsen AB, Zauber AG, Rutter CM, Naber SK, Doria-Rose VP, Pabiniak C,
PharmFIT: assessing the feasibility of a pharmacy-based fecal Johanson C, Fischer SE, Lansdorp-Vogelaar I, Kuntz KM. Estimation of ben-
immunochemical test kit distribution program to increase efits, burden, and harms of colorectal cancer screening strategies: model-
colorectal cancer screening access ing study for the US Preventive Services Task Force. JAMA.
Mary Wangen1, Catherine Rohweder1, Rachel Ceballos2, Renee 2016;315(23):2595-2609.
Ferrari1, Rachel Issaka2, Dan Reuland1, Jennifer Richmond1, Sara Rubio 7. US Preventive Services Task Force, Bibbins-Domingo K, Grossman DC,
Correa1, Parth Shah2, Stephanie Wheeler1, Alison Brenner1 et al. Screening for colorectal cancer: US Preventive Services Task Force
1
University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; 2Fred recommendation statement. JAMA. 2016;315(23):2564-2575.
Hutchinson Cancer Research Center, Seattle, WA, USA 8. Beebe J. Rapid assessment process: an introduction. Lanham: AltaMira Press; 2001.
Correspondence: Mary Wangen ([email protected]) 9. Krippendorff K. Content Analysis: an introduction to its methodology
Implementation Science 2020, 15(Suppl 2):A170 2ed. Thousand Oaks, CA: Sage; 2004.
10. Feldstein AC, Glasgow RE. A practical, robust implementation and
Background sustainability model (PRISM) for integrating research findings into
Rural populations have lower rates of colorectal cancer (CRC) screen- practice. Jt Comm J Qual Patient Saf. 2008;34(4):228-243.
ing [1-3] and sub-optimal access to preventive care services [4]. In 11. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu
North Carolina, geospatial analyses have revealed that sub-optimal MM, Proctor EK, Kirchner JE. A refined compilation of implementation
access to health services in rural regions significantly related to strategies: results from the Expert Recommendations for Implementing
higher rates of CRC mortality in rural “hotspots” [4]. As such, identify- Change (ERIC) project. Implement Sci. 2015;10(1):21.
ing alternative health care settings in rural areas that could deliver
CRC screenings may be one way to alleviating this health inequity. A171
Pharmacies may be an opportune setting to distribute fecal immuno- A systematic review of the barriers and enablers to
chemical test (FIT) kits for CRC screening in rural areas as one way to implementation of menu labelling interventions from a food
improve access to CRC screening in communities with poorer access service industry perspective
to traditional care delivery settings. FIT kits are a guideline- Claire Kerins1, Sheena McHugh2, Jennifer McSharry1, Catherine
recommended screening test that patients can complete at home [5- Hayes3, Caitlin M. Reardon4, Fiona Geaney2, Ivan J. Perry2, Suzanne
7]. This formative work will assess the feasibility and acceptability of Seery5, Colette Kelly1
1
delivering FIT kits for CRC screening in pharmacy settings (PharmFIT). National University of Ireland Galway, Galway, Ireland; 2University
Materials and Methods College Cork, Cork, Ireland; 3Trinity College Dublin, Dublin, Ireland; 4Ann
We are conducting semi-structured interviews with key informants to Arbor VA Center for Clinical Management Research, Ann Arbor, MI, USA;
5
elicit: 1) knowledge, attitudes, and perceptions of a PharmFIT interven- National Institute for Prevention and Cardiovascular Health, Galway,
tion; 2) barriers and facilitators to implementing PharmFIT; and 3) recom- Ireland
mendations for implementation strategies that would support successful Correspondence: Claire Kerins ([email protected])
delivery of a PharmFIT intervention. We are interviewing 12 primary care Implementation Science 2020, 15(Suppl 2):A171
Background are developing and implementing a two-phase screening study to

Menu labelling has gathered growing public and legislative support in identify older patients at-risk for malnutrition and food insecurity in the
response to the increased consumption of foods outside the home and emergency department (ED) and connect them to a community-based
the associated risks of overweight and obesity. A recent systematic re- organization (CBO) to address social needs.
view has shown menu labelling effects consumer food choice and the Materials and Methods
food industry behaviour [1]. Several countries have introduced menu la- During Phase 1, multiple stakeholder perspectives (i.e., ED, patient,
belling policies on a voluntary or mandatory basis; however, challenges CBO) were collected on how best to implement a sustainable ED-
to implementation have arisen (e.g. poor uptake, inaccurate nutritional based screening process that considers the complexities and rapid
information). The aim of this systematic review was to synthesise the pace of the ED. ED stakeholders included registered nurses (RNs),
evidence on the barriers and enablers to menu labelling implementa- nursing assistants (NAs), social workers (SWs). To understand ED per-
tion from the food industry perspective. spectives, research staff conducted semi-structured interviews (SSIs)
Materials and Methods with interview guides developed using the Consolidated Framework
The review adopted the ‘best fit’ framework synthesis approach, de- for Implementation Research (CFIR) [7] Guides included constructs
signed for policy urgent questions [2]. No restrictions applied to pub- from four CFIR domains (i.e., intervention characteristics, inner set-
lication type, study design, data collection method, language or ting, characteristics of individuals, process). SSIs were transcribed and
publication year. At least two independent reviewers performed analyzed using framework-guided rapid analysis [8-9]. To understand
study selection, data extraction and quality appraisal. A combination the patient perspective, research staff tested screening questions
of deductive coding, using the Consolidated Framework for Imple- with 75 older patients to identify how many are at-risk, and gauge
mentation Research [3] as the a priori framework, and inductive ana- receptivity to screening and referrals within the ED. Research staff
lysis, using secondary thematic analysis were undertaken. worked with a CBO to understand their perspective on data sharing,
Results referral pathways and “closing-the-loop.”
The overall process led to the construction of an adapted version of Results
the CFIR. Of the 2,806 articles identified, 17 studies met the eligibility Nine SSIs (n=3 RNs, n=2 NAs, n=4 SWs) were analyzed. Common themes
criteria. Most frequently cited barriers were coded to the CFIR con- related to constructs from CFIR domains were identified, including build-
structs ‘Consumer Needs & Resources’ (e.g. lack of customer demand ing the screener into the EHR, comparing to existing screening processes,
and understanding) and ‘Compatibility’ (e.g. lack of standardised rec- and educating ED staff. A critical finding was that NAs, not RNs, should
ipes, limited space on menus). Commonly cited facilitators were coded screen patients. From the sample of patients screened, approximately
to the CFIR constructs ‘Relative Advantage’ (e.g. improved business 35% were positive for malnutrition, 18% for food insecurity and 8% for
image/reputation) and ‘Consumer Needs & Resources’ (e.g. customer both. Patients were receptive to being screened in the ED and indicated
demand, enabling healthier food choices). Relationships between con- they would be willing to receive help connecting to CBOs. The CBO in-
structs (across and within domains) were also evident. The revised formed the development of referral workflows, particularly bidirectional
framework, based on the final list of constructs from the deductive and communication and datapoints to facilitate data sharing.
inductive coding, maintained many of the essential elements of the Conclusions
CFIR but a number of (sub)constructs with no supporting data were re- The stakeholders’ perspectives informed the workflows that will be
moved and newly developed constructs incorporated. implemented and evaluated in Phase 2, including NAs screening,
Conclusions SWs connecting patients to the CBO, and the CBO sharing updates
Findings from this review provide a foundation for selecting and tai- on the status of services provided to patients.
loring implementation strategies to improve adoption, implementa-
tion, sustainment, and scale-up of menu labelling interventions.
Moreover, in refining the CFIR, this review provides a theoretical con- References
tribution to help advance the field of implementation science. 1. Pereira GF, Bulik CM, Weaver MA, Holland WC, Platts-Mills TF. Malnutrition
among cognitively intact, noncritically ill older adults in the emergency
References department. Ann Emerg Med. 2015;65(1):85-91
1. Shangguan S, Afshin A, Shulkin M, Ma W, Marsden D, Smith J, Saheb- 2. Saka B, Kaya O, Ozturk GB, Erten N, Karan MA. Malnutrition in the elderly
Kashaf M, Shi P, Micha R, Imamura F, Mozaffarian D; Food PRICE Project. a and its relationship with other geriatric syndromes. Clin Nutr.
meta-analysis of food labeling effects on consumer diet behaviors and 2010;29(6):745-748
industry practices. Am J Prev Med. 2019;56(2):300-314. 3. Agarwal E, Ferguson M, Banks M, Batterham M, Bauer J, Capra S, Isenring E.
2. Kerins C, McSharry J, Hayes C, Perry IJ, Geaney F, Kelly C. Barriers and Malnutrition and poor food intake are associated with prolonged hospital
facilitators to implementation of menu labelling interventions to support stay, frequent readmissions, and greater in-hospital mortality: results from
healthy food choices: a mixed methods systematic review protocol. Syst the Nutrition Care Day Survey 2010. Clin Nutr. 2013;32(5):737-745
Rev. 2018; 7(1):88. 4. Burks CE, Jones CW, Braz VA, Swor RA, Richmond NL, Hwang KS,
3. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Hollowell AG, Weaver MA, Platts-Mills T. Risk factors for malnutrition
Fostering implementation of health services research findings into among older adults in the emergency department: a multicenter study. J
practice: a consolidated framework for advancing implementation Am Geriatr Soc. 2017;65(8):1741-1747.
science. Implement Sci. 2009; 4(1):50. 5. Agarwal E, Miller M, Yaxley A, Isenring E. Malnutrition in the elderly: a
narrative review. Maturitas. 2013;76(4):296-302
6. National Academies of Sciences E, and Medicine (NASEM). Meeting the
A172 dietary needs of older adults: Exploring the impact of the physical, social,
Multiple stakeholders’ perspectives on screening older adults for and cultural environment: Workshop summary. Washington, DC: The
malnutrition and food insecurity in an emergency department National Academies Press; 2016.
setting 7. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC.
Jessa Engelberg1, Andrea Morris1, Aileen Aylward2, Rayad Shams2, Tim Fostering implementation of health services research findings into
Platts-Mills2 practice: a consolidated framework for advancing implementation
1
West Health Institute, La Jolla, CA, USA; 2University of North Carolina, science. Implement Sci. 2009; 4:50.
Chapel Hill, NC, USA 8. Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ,
Correspondence: Jessa Engelberg ([email protected]) Midboe AM. Comparison of rapid vs in-depth qualitative analytic
Implementation Science 2020, 15(Suppl 2):A172 methods from a process evaluation of academic detailing in the Veterans
Health Administration. Implement Sci. 2019;14(1):11.
Background 9. Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the
Malnutrition is common among older adults and contributes to poor consolidated framework for implementation research (CFIR) to produce
health and premature death [1-3]. Malnutrition is a complex condition actionable findings: a rapid-cycle evaluation approach to improving im-
with medical and social risk factors [4-6], including food insecurity. We plementation. Implement Sci. 2017;12:15.
A173 Taking Texas Tobacco Free is one such tobacco-free workplace program
Effective implementation strategies for male engagement in a that has been successfully implemented within hundreds of behavioral
program for Zambian couples experiencing IPV and substance mis-use health clinics in Texas (www.takingtexastobaccofree. 65(8):1741-1747
Laura Eise1, Stephanie Skavenski van Wyk2, Jeremy C. Kane2, Kristina com) [2-5]. In 2017, we were funded to expand the program to dedicated
Metz2, Laura K. Murray2 substance use treatment centers.
1
University of Washington, Seattle, WA, USA; 2Johns Hopkins University, Materials and Methods
Baltimore, MD, USA Taking Texas Tobacco Free implementation includes a tobacco-free work-
Correspondence: Laura Eise ([email protected]) place policy along with the provision of education and specialized provider
Implementation Science 2020, 15(Suppl 2):A173 training to enable the institution of regular tobacco-use assessments and
treatment provision or referral for tobacco dependence. A mixed-methods,
Background formative evaluation process is used to understand clinic-specific facilitators
The link between violence and substance use is well established. How- and potential barriers, which guides the implementation strategies within di-
ever, the violence literature has largely evaluated prevention programs, verse contexts. Consultation, practical guidance, and treatment resources are
or those focused on women only. The need to include male counterparts provided and mechanisms for program sustainability are emphasized.
is recognized, yet challenging. Evidence indicates that males disconnect Results
from health-care services at high rates and, furthermore, information spe- Enrolled clinics (N=8) serve ~70,000 unique consumers annually, includ-
cific to low- and middle-income countries (LMIC) is scarce. In order to ing some special groups (e.g., sexual minorities, women with children).
offer effective care, we must better understand how to engage men in Thus far, Taking Texas Tobacco Free has educated 1,119 professionals
these situations, which may include economic barriers, pervasive societal through 61 discrete sessions and reached 92,521 individuals through pas-
attitudes, and self-stigma to access available services. sive material dissemination. Each clinic has adopted a 100% tobacco-free
Materials and Methods workplace policy, integrated tobacco-use assessments into routine prac-
We recently completed a randomized controlled trial (RCT) of a trans- tice, delivered evidence-based interventions, and dispensed nicotine re-
diagnostic psychotherapy (the Common Elements Treatment Approach; placement therapies to consumers and staff. Recruitment is ongoing.
CETA) delivered to Zambian couples who indicated recent intimate Conclusions
partner violence (IPV), as well as male substance misuse. In order to re- Taking Texas Tobacco Free is an effective comprehensive tobacco con-
cruit participants within a concentrated period (i.e., 12 weeks), we trol program that has proven successful in implementing tobacco-free
worked closely with a local partner that had successfully engaged men workplace policies, training providers in tobacco cessation interventions
in an alcohol mis-use program. The partner utilized a multi-tiered and in integrating those interventions into regular clinical practice
process of community word-of-mouth and engagement by respected within substance use treatment clinics. This presentation will describe
peers, with an emphasis on a non-stigmatizing process of screening. the program, participating clinics, data-based strategies used to tailor
Results implementation within each setting, accomplishments to date (e.g.,
Throughout the study, there were high rates of male engagement knowledge gained by staff and clinicians, tobacco-use assessments pro-
and significant treatment effects both for the reduction of violence vided, pre- versus post-implementation changes in clinician behavior),
and alcohol use. To gain a more in-depth understanding of the RCT and lessons learned during the implementation process that can guide
findings, including the male engagement and retention, we qualita- program dissemination in other settings and states.
tively explored mechanisms of behavior change related to male per-
petration of IPV. We sampled adult men and women from the 123 References
couples randomized to the intervention arm. We conducted 30 first 1. Marynak K, VanFrank B, Tetlow S, Mahoney M, Phillips E, Jamal Mbbs A,
round interviews (16 women; 14 men) and then re-interviewed 20 Schecter A, Tipperman D, Babb S. Tobacco cessation interventions and
participants (13 women; 7 men). In addition, we conducted 4 focus smoke-free policies in mental health and substance abuse treatment facilities
groups (2 women; 2 men). We also analyzed an implementation log — United States, 2016. MMWR Morb Mortal Wkly Rep. 2018;67:519–523.
maintained by study staff documenting engagement and retention 2. Correa-Fernández V, Wilson WT, Kyburz B, O’Connor DP, Stacey T, Wil-
strategies, challenges, and successes throughout the study. liams T, Lam CY, Reitzel LR. Evaluation of the Taking Texas Tobacco Free
Conclusions Workplace Program within behavioral health centers. Transl Behav Med.
Based on the results of this data collection and analysis, our poster 2019;9(2):319-327
will highlight the successful implementation strategies used in this 3. Correa-Fernández V, Wilson WT, Shedrick DA, Kyburz B, L Samaha H, Sta-
RCT on IPV and substance use, present the data documenting the cey T, Williams T, Lam CY, Reitzel LR. Implementation of a tobacco-free
engagement and retention of male participants, and summarize the workplace program at a local mental health authority. Transl Behav Med.
qualitative responses from men themselves that highlight key facilita- 2017;7(2):204-211.
tors and barriers to engagement. 4. Samaha HL, Correa-Fernández V, Lam C, Wilson WT, Kyburz B, Stacey T,
Trial Registration: ClinicalTrials.gov NCT02790827 Williams T, Reitzel LR. Addressing tobacco use among consumers and
staff at behavioral health treatment facilities through comprehensive
workplace programming. Health Promot Prac. 2017;18(4):561-570.
A174 5. Centers for Disease Control and Prevention (CDC). Promising policies and
Changing policy and practice in substance use treatment clinics practices to address tobacco use by persons with mental and substance
through the implementation of a tailored, comprehensive tobacco use disorders: Texas Provides NRT as part of a range of tobacco cessation
free workplace program measures in mental health treatment settings. 2018. https://www.cdc.gov/
Lorraine Reitzel1, Bryce Kyburz2, Isabel Leal1, Kathy Le1, Virmarie Correa- tobacco/disparities/ promising-policies-and-practices/pdfs/osh-behavioral-
Fernandez1, Teresa Williams2, Daniel O’Connor1, Ezemenari Obasi1, health-promising-practices-20160709-p.pdf. Accessed 7 March 2019.
Kathleen Casey2
1
University of Houston, Houston, TX, USA; 2Integral Care, Austin, TX, USA
Correspondence: Lorraine Reitzel ([email protected]) A175
Implementation Science 2020, 15(Suppl 2):A174 Conducting a process evaluation of a statewide opioid prescribing
policy: applying the Consolidated Framework for Implementation
Background Research across all phases of data collection and analysis
Despite elevated tobacco use rates among individuals in treatment for Natalie Blackburn1, Elizabeth Joniak-Grant2, Maryalice Nocera3, Jada
substance use disorders in Texas, only 70.2% of treatment clinics screen Walker3, Shabbar Ranapurwala3
1
consumers for tobacco use, 55.4% provide cessation counseling, 24% UNC-Chapel Hill, Chapel Hill, NC, USA; 2National Coalition of Independent
offer nicotine replacement therapies, and 34.3% have a tobacco-free Scholars, Battleboro, VT, USA; 3UNC Injury Prevention Research Center,
workplace policy [1]. Comprehensive tobacco-free workplace programs University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
that include all of these evidence-based strategies are known to be ef- Correspondence: Natalie Blackburn ([email protected])
fective in reducing tobacco use among this vulnerable population [1]. Implementation Science 2020, 15(Suppl 2):A175

Opioid dependence and overdose are serious public health concerns Capacity to Treat Co-Occurring Chronic Pain and Opioid Use Disorder
in the United States [1]. States have sought to identify and imple- (CAP-POD) questionnaire items were developed over a 2-year process
ment policy interventions to address these growing public health including literature review, semi-structured interviews, expert panel
problems. One policy intervention identified has been to set prescrib- review, and pilot testing. A national sample of PCPs (MD, DO, NP, PA)
ing limits such that physicians prescribe no more than five days’ sup- were recruited via email to complete an online survey that included
ply for post-surgical acute pain in an effort to reduce patient misuse the 44-item CAP-POD questionnaire. Response options ranged from
and overdose [2]. As such in 2017 North Carolina passed the 1 (strongly disagree) to 7 (strongly agree). CAP-POD items were ana-
Strengthen Opioid Misuse Prevention (STOP) act which limits pre- lyzed for dimensionality and inter-item reliability. We compared
scriptions to three to five-day supplies of opioids for acute post- mean scores across provider characteristics (education, setting, years’
surgical pain. The purpose of this study is to understand the barriers experience) to identify potential gaps in capacity.
and facilitators in the passage and implementation of the STOP act Results
in North Carolina. 509 PCPs from across the US completed the questionnaire. Principal com-
Materials and Methods ponent analysis resulted in a 22-item questionnaire. Twelve more items
Three groups were identified as key figures in the passage and im- were removed because of their influence on coefficient alphas, resulting
plementation of the STOP act: government officials, hospital adminis- in a 10-item questionnaire with 4 domains: 1) Motivation to Treat patients
trators, and opioid prescribers. Using the Consolidated Framework with chronic pain and OUD (α =.87, M=3.49,SD=1.48); 2) Trust in Evidence
for Implementation Science (CFIR) [3] we developed three separate (α =.87, M=5.67, SD=1.03); 3) Assessing Risk (α =.82, M=5.45, SD=1.19);
interview guides to be administered in one-on-one interviews. Two and 4) Patient Access to therapies (α =.79; M=3.06, SD=1.47). Mean scores
researchers developed the guides by reviewing all constructs within across the four scales differed significantly (p<.001).
the CFIR and identifying construct-derived questions that would be Conclusions
most salient for each interview group. Questions were adapted to fit We developed a short, 10-item questionnaire that assesses the cap-
a policy context as well as fitting the role of the individual in policy acity of PCPs to implement best practice recommendations for the
implementation. treatment of co-occurring chronic pain and OUD. The questionnaire
Results and scales demonstrated adequate validity and good inter-item reli-
This research project is currently in the data collection stage. Inter- ability. PCPs reported moderate trust in evidence of treatments for
views will be conducted from March to May 2019; analysis will begin co-occurring chronic pain and OUD, and in their ability to identify pa-
in June 2019. In addition to using the CFIR to design the interview tients at risk. Conversely, they had low desire to treat these patients,
guides, the CFIR will inform our coding of interview data and the de- and see their patients’ access to relevant services as suboptimum.
velopment of analytical themes. Interviews will be analyzed across These data imply a service shortfall that will likely require fixing with
groups in order to summarize the perspectives and identify unifying additional training, service design, and incentives. The questionnaire
or diverging ideas that may inform how the law is being imple- provides a brief, validated evaluation tool for such interventions.
mented in the state.
Conclusions
Few studies have used the CFIR throughout the data collection A177
process from development of interview guides to development of Tailoring practice facilitation to optimize alcohol-related care in
qualitative codebooks and conducting analysis. Given the growing hepatology clinics: barriers and facilitators and feedback on an
number of States proposing laws to limit opioid prescribing, under- implementation intervention
standing the experience of North Carolina in implementing the STOP Ann Marie Roepke1,2, Madeline Frost1, George Ioannou1,2, Judith
act will inform how states might support their key partners for more Tsui2,3, Jennifer Edelman4, Bryan Weiner2, Amy Edmonds1,2, Emily
effective policy implementation. Williams1,2,5
1
VA Puget Sound Health Care System, Seattle, WA, USA; 2University of
References Washington, Seattle, WA, USA; 3Harborview Medical Center, Seattle, WA,
1. Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in drug and USA; 4Yale University, New Haven, Connecticut, USA; 5Seattle-Denver
opioid overdose deaths—United States, 2000–2014. Am J Transplant. COIN, Seattle, WA, USA
2016;16(4):1323-1327. Correspondence: Ann Marie Roepke ([email protected])
2. Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids Implementation Science 2020, 15(Suppl 2):A177
for chronic pain—United States, 2016. JAMA. 2016;315(15):1624-1645.
3. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC. Background
Fostering implementation of health services research findings into Unhealthy alcohol use exacerbates and complicates treatment of
practice: a consolidated framework for advancing implementation chronic liver disease [1]. Yet, evidence-based alcohol-related care is
science. Implement Sci. 2009; 4:50. inconsistently delivered in hepatology clinics [2]. Informed by re-
search supporting practice facilitation as an effective implementation
strategy in primary care, we aim to tailor practice facilitation to im-
A176 plement evidence-based alcohol-related care in four Veterans Affairs
Development and psychometric testing of the capacity to Treat (VA) hepatology clinics [3]. Here we describe barriers and facilitators
Co-Occurring Chronic Pain and Opioid Use Disorder (CAP-POD) garnered from qualitative interviews with key stakeholders at 2 of 4
Questionnaire clinics to inform intervention tailoring.
Allyson Varley1, Stefan Kertesz1, Andrea Cherrington1, Aerin Materials and Methods
deRussy2, April Hoge2, Kevin Riggs1, Peter Hendricks1 We recruited key stakeholders (n=23) including clinicians (MD, NP),
1
University of Alabama at Birmingham, Birmingham, AL, USA; clinical staff (RN, LPN, MSW), and administrators responsible for car-
2
Birmingham VA Medical Center, Birmingham, AL, USA ing for Veterans with liver conditions. Semi-structured qualitative in-
Correspondence: Allyson Varley ([email protected]) terviews were developed using the Consolidated Framework on
Implementation Science 2020, 15(Suppl 2):A176 Implementation Research (CFIR) and specifically focused on under-
standing outer and inner setting and individual domains [4]. We elic-
Background ited stakeholders’: (1) context for, experiences with, and perspectives
Patients with the combination of chronic pain and opioid use dis- about providing care to Veterans with liver conditions and unhealthy
order have unique needs and may present a challenge for clinicians alcohol use; and (2) feedback regarding a practice facilitation inter-
and health care systems. Primary care providers’ (PCPs) capacity to vention. Rapid content analysis was used to extract relevant themes.
deliver high quality, evidence-based care for this important subpopu- Results
lation is unknown. This study’s objective was to develop and test a Qualitative interviews highlighted barriers to and facilitators of pro-
survey of PCP capacity to treat co-occurring chronic pain and OUD. viding alcohol-related care and tailoring practice facilitation. Barriers
included lack of systematic alcohol screening procedures; variability Results

in clinicians’ knowledge, comfort, and interest in providing evidence- This work revealed that implementation researchers and entrepreneurs
based treatments (e.g., medications for alcohol use disorder); per- had different perspectives on: criteria for rollout readiness, return on in-
ceived inadequate linkage with specialty addiction treatment and/or vestment, risk tolerance, and product goals. Implementation re-
behavioral health; and challenges related to staffing time/availability. searchers focus on empirical evidence of innovation benefit,
Facilitators included system- and clinic-level leadership support, his- minimizing risk and unanticipated consequences, and carefully se-
tories of successful quality improvement efforts, staff who are well- quenced steps in implementation. Entrepreneurs focused on market
prepared to serve as clinical champions, consensus regarding the im- demand, innovation cost, numbers expected to benefit from the
portance of addressing alcohol use, enthusiasm for several planned innovation, and infrastructure and payment required for sustainment.
practice facilitation elements, and cohesive teams. Consistent with Conclusions
the CFIR, findings from 23 liver clinic staff suggest that a practice fa- Harnessing the synergy between these disciplines can advance full
cilitation intervention can capitalize and build on existing setting and realization of the benefits of biomedical research health care and popu-
individual characteristics. lation health. Both fields face the reality that the number of discoveries
Conclusions needing translational support greatly exceeds available funding and ab-
Specifically, the intervention should build on existing leadership sup- sorptive capacity. Innovative approaches, infrastructure development,
port, enthusiasm, team cohesiveness, and successful past quality im- and training are required to leverage the yet-untapped synergy be-
provement efforts and include: (1) assistance integrating tween these fields. We identify a number of mechanisms for advancing
standardized alcohol screening into clinic flow; (2) training and on- synergy between these two fields–an exemplar of team science.
going support regarding evidence-based care for unhealthy alcohol
use; and (3) linkages with or internal capacity building for behavioral
health or specialty addictions treatment. A179
Implementing across an integrated health care system and a state
References criminal justice system – lessons from a peer support intervention
1. Fuster D, Samet JH. Alcohol use in patients with chronic liver disease. N for veterans leaving incarceration
Engl J Med. 2018; 379(13):1251-1261. D. Keith McInnes1, Justeen Hyde1,2, Thomas Byrne3, Beth Ann Petrakis1,
2. Owens MD, Ioannou GN, Tsui JI, Williams EC. Receipt of alcohol-related Vera Yakovchenko1
1
care among patients with HCV and unhealthy alcohol use. Drug Alcohol Department of Veterans Affairs, Boston, MA, USA; 2Center for Healthcare
Depend .2018; 188:79-85. Organization and Implementation Research, Boston, MA, USA; 3Boston
3. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of University, Boston, MA, USA
practice facilitation within primary care settings. Ann Fam Med. Correspondence: D. Keith McInnes ([email protected])
2012;10(1):63-74. Implementation Science 2020, 15(Suppl 2):A179
4. Damschroder LJ, Aron DC. Keith RE, Kirsh SR, Alexander JA, Lowery JC.
Fostering implementation of health services research findings into Background
practice: a consolidated framework for advancing implementation Veterans just released from incarceration (“reentry veterans”) experi-
science. Implement Sci. 2009;4:50. ence barriers to housing and health services, which heightens risk for
homelessness, recidivism, morbidity and mortality [1]. The Veterans
Health Administration (VHA) employs peer support specialists, but
A178 does not have dedicated peers supporting the multi-faceted needs
Implementation science and entrepreneurship: harnessing synergy of reentry veterans. We developed and implemented the VHA’s first
for discovery uptake peer-support [2] initiative for reentry veterans, the Post-Incarceration
Enola Proctor1, Emre Toker2, Rachel Tabak1, Cole Hooley3, Virginia Engagement (PIE) program.
McKay1 Materials and Methods
1
Brown School, Washington University in St. Louis, St. Louis, MO, USA; We used a Facilitation strategy to implement PIE in Massachusetts.
2
Arizona State University, Tempe, AZ, USA; 3Brigham Young University, External facilitation included developing an intervention manual,
Provo, UT, USA worksheets, training curriculum, and hiring veteran peers. Peers met
Correspondence: Enola Proctor ([email protected]) weekly with reentry veterans to address life priorities, and support
Implementation Science 2020, 15(Suppl 2):A178 community reintegration in 3 areas: linkage to services, skill building,
and social support. External facilitation also involved stakeholder en-
Background gagement with Massachusetts’ Department of Correction (DOC),
Implementation science and social entrepreneurship share the goal Mental Health, and Veterans Services; network development with
of accelerating the uptake of medical discoveries for widespread use community-based service providers; and marketing presentations to
in clinical and community healthcare. This paper reports infrastruc- VHA regional homelessness programs and to veteran inmates in DOC
ture and activities within a CTSA program to advance synergy be- facilities. Internal facilitation, led by a social worker champion, in-
tween these two fields. This work is based on the assumptions that volved educating service line chiefs at Massachusetts’ VHA medical
implementation science can benefit from entrepreneurship’s em- centers about the benefits of the PIE program.
phasis on market demand, while entrepreneurship can benefit from Results
implementation science’s emphasis on data, models, and context— High levels of collaboration were achieved between with the Depart-
particularly the policy, social, and organizational context of ment of Veterans affairs central office justice programs and a coali-
healthcare. tion of state agencies and community organizations. PIE served 30
Materials and Methods reentry veterans released from 6 DOC prisons and 3 county jails.
Our CTSA conducted activities to identify challenges and areas of Peers had over 200 encounters with these veterans. PIE reentry vet-
complementarity between implementation researchers and entre- erans had a higher likelihood than comparison veterans of linkage to
preneurs. First, we held a series of meetings between implemen- substance use treatment (80% versus 19%, respectively, P<0.001) and
tation researchers and entrepreneurs, the purpose of which was mental health care (87% versus 64%, respectively, borderline signifi-
to identify and map shared and distinctive approaches regarding cant). They were more likely than comparison veterans to access
criteria for roll-out readiness, metrics for assessing return on in- VHA homelessness services, such as the domiciliary inpatient pro-
vestment, roll-out processes, risk tolerance, and priority products. gram (53.3% versus 2.7%, respectively, P<0.001) and short-term
Second, we convened an IdeaBounce@ experience, an event in emergency beds (26.7% versus 5.4%, respectively, P=0.05). Though
which implementation researchers “pitched” their innovations to not statistically significant, trends suggested reentry veterans had
an audience of entrepreneurs and received feedback. A qualita- greater access, than comparison veterans, to transitional grant and
tive researcher observed both activities, taking notes and synthe- per diem housing (26.7% versus 25.4%, respectively) and to federal
sizing observations. housing vouchers (6.7% versus 3.7%, respectively).
Conclusions 4. Pathman DE, Steiner BD, Jones BD, Konrad TR. Preparing and retaining
In summary, a facilitation strategy contributed to the implementation rural physicians through medical education. Acad Med. 1999;74:810–820.
of a reentry veteran peer support program which shows promise in 5. Smith TA, Adimu TF, Martinez AP, Minyard K. Selecting, adapting, and
improving access to housing and health services. implementing evidence-based interventions in rural settings: an analysis
of 70 community examples. J Health Care Poor Underserved.
References 2016;27(4A):181-193.
1. Visher CA, Travis J. Transitions from prison to community: understanding 6. Peters MDJ, Godfrey C, McInerney P, Baldini Soares C, Khalil H, Parker D.
individual pathways. Annu Rev Sociol. 2003;29:89–113. Chapter 11: Scoping Reviews. In: Aromataris E, Munn Z, eds. Joanna
2. Chinman M, George P, Dougherty RH, Daniels AS, Ghose SS, Swift A, Briggs Institute Reviewer’s Manual. The Joanna Briggs Institute; 2017.
Delphin-Rittmon ME. Peer support services for individuals with serious https://reviewersmanual.joannabriggs.org. Accessed 1 Feb 2019
mental illnesses: assessing the evidence. Psychiatr Serv. 2014; 65(4):429–
41.
A181
Identifying determinants of implementation of the Cornerstone
A180 Intervention to develop a user-centered implementation manual
Implementation of evidence-based mental health interventions in Danielle Adams1, Andrea Cole2, Michelle Munson3, Curtis McMillen1,
rural settings: a scoping literature review Victoria Stanhope3
1
Christopher Weatherly, Meagan Pilar University of Chicago, Chicago, IL, USA; 2New York State Psychiatric
Brown School, Washington University in St. Louis, St. Louis, MO, USA Institute, New York, NY, USA; 3New York University, New York, USA
Correspondence: Christopher Weatherly ([email protected]) Correspondence: Danielle Adams ([email protected])
Bridging the gap between rural and urban mental health services re- Transition-age youth have elevated rates of mental disorders, and
quires developing creative solutions to complex challenges that are often do not receive services. Few mental health interventions have
unique to rural areas. Previous research has documented numerous been developed for older youth in transition, and even fewer have
mental health disparities in rural settings, including higher rates of been found to be effective over the transition to adulthood. Corner-
depression and suicide [1-2], and limited resources and barriers to stone, a theoretically-guided intervention has shown promise for ad-
care restrict access to mental health services for rural residents [3-4]. dressing the mental health needs of this group as they emerge into
Despite the demonstrated need for intervention, there are significant adulthood [1]. Cornerstone provides case management, trauma-
challenges associated with implementing evidence-based interven- focused cognitive behavioral therapy, mentoring/peer support, and
tions in rural community settings [5]. However, little is known about community-based in-vivo practice to address stigma and mental
the current state of mental health implementation in rural settings. health symptoms, and practical skill development to improve the
This scoping review thus aims to address this gap by providing a sys- transition to independence among TAY with mental health condi-
tematic overview of how evidence-based mental health interventions tions [2]. Using the Consolidated Framework for Implementation Re-
are being implemented within rural community settings. search (CFIR) [3], this study examined determinants of
Materials and Methods implementation of Cornerstone with the goal of creating an imple-
The scoping review is structured according to Peters et al’s frame- mentation manual to guide real-world effectiveness trials and scal-
work for conducting scoping studies [6]. We searched the following ability efforts.
databases: PubMed, CINAHL, PsychINFO, EMBASE, SCOPUS, Web of Materials and Methods
Science, ClinicalTrials.gov, and the Cochrane Library. The three search Within a Hybrid Type 2 trial, investigators developed a semi-
strings used for this review included variations of “mental health,” structured interview protocol using implementation strategy domains
“implementation,” and “rural.” Inclusion criteria: 1) empirical study in- as a framework [4]. Face-to-face interviews were conducted with
volving the implementation of a mental health intervention in a rural clinic staff (n = 8) and state-level leadership (n = 3), and research
setting; 2) English language; and 3) peer-reviewed journal publica- staff (n = 1) on determinants of implementation for Cornerstone,
tion. No restrictions were placed on year of publication, sample size, such as planning, training, and supervision. Using grounded theory
or research design. Screening and review of articles will be carried with sensitizing concepts, multiple coders analyzed the data using
out by two reviewers. A third reviewer will be involved as needed for constant comparison. Iterative discussion(s) occurred over six months
consensus. We will assess and review findings through both tabular until saturation was met.
and thematic analyses. Results
Results Using the CFIR [3], we created a comprehensive review of implemen-
We are currently in the process of screening abstracts. In April-May tation determinants of the Cornerstone intervention, as well as a re-
2019, we will review and extract full texts. We will present the yields, view of contextual information (e.g., state policy reforms) from state-
characteristics of included articles, and a description of evidence- level stakeholders which may impact future scalability and sustain-
based mental health interventions being implemented and tailored ability of the intervention. Outer setting themes converged around
to the unique rural context. We will also describe the implementation the external policy context and incentives, with respondents discuss-
strategies, adaptation methods, and outcomes associated with these ing value-based payment and the importance of tracking non-
studies. billable tasks of mentors. Process themes pointed to important areas
Conclusions of planning: integration of mentors within the clinic, regular team
This project is intended to provide an overview of the current state check-ins, and the increased use of technology by mentors. Partici-
of mental health implementation research in rural settings to identify pants qualitatively reported high acceptability and feasibility for the
gaps in previous research and identify areas for future work. cornerstone intervention and its components.
Conclusions
References Results will be combined with user-centered design approaches, such
1. Eberhardt MS, Pamuk ER: The importance of place of residence: as the simplification principle [5], to develop a Cornerstone Imple-
examining health in rural and nonrural areas. Am J Public Health. mentation Manual that will assist us in moving toward testing effect-
2004;94:1682–1686. iveness of a much-needed intervention for TAY.
2. Simmons LA, Braun B, Charnigo R, Havens JR, Wright DW. Depression Trial Registration ClinicalTrials.gov NCT02696109
and poverty among rural women: a relationship of social causation or
social selection? J Rural Health. 2008;24:292–298. References
3. Orloff TM, Tynmann B. Rural health: an evolving system of accessible 1. Cole AR, Munson MR, Ben David S, Sapiro B, Railey J, Stanhope V.
services. Washington, DC: National Governors’ Association. 1995. Feasibility, acceptability, and preliminary impact of the Cornerstone
mentoring program. Paper presented at the 10th Annual Conference on 2. Langi FLFG, Balcazar FE. Risk factors for failure to enter vocational
the Science of Dissemination and Implementation, Arlington, VA; 2017. rehabilitation services among individuals with disabilities. Disabil Rehabil.
2. Munson MR, Cole A, Stanhope V, Marcus SC, McKay M, Jaccard J, Ben- 2017;39(26):2640-2647. doi:10.1080/09638288. 2016.1236410.
David S. Cornerstone program for transition-age youth with serious men- 3. Bergmark M, Bejerholm U, Markström U. Critical components in
tal illness: study protocol for a randomized controlled trial. Trials. implementing evidence-based practice: a multiple case study of Individ-
2016;17(1):537-550. ual Placement and Support for people with psychiatric disabilities. Soc
3. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Policy Adm. 2018;52(3):790-808. doi:10.1111/spol.12243.
Fostering implementation of health services research findings into 4. Bazelon Center for Mental Health Law. Advances in Employment Policy
practice: a consolidated framework for advancing implementation for Individuals with Serious Mental Illness. Washington DC; 2018. http://
science. Implement Sci. 2009; 4(1):50. www.bazelon.org/wp-content/uploads/2018/10/Supported-Employment-
4. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, Report_Oct-2018.pdf. Accessed 9 March 2019.
Glass JE, York JL. A compilation of strategies for implementing clinical 5. Interdepartmental Serious Mental Illness Coordinating Committee. The
innovations in health and mental health. Med Care Res Rev. way forward: federal action for a system that works for all people living
2012;69(2):123-157. with smi and sed and their families and caregivers. Washington DC;
5. Lyon AR, Koerner K. User‐centered design for psychosocial intervention 2017. https://store.samhsa.gov/system/files/pep17-ismicc-rtc.pdf.
development and implementation. Clin Psychol. 2016;23(2):180-200. Accessed 9 March 2019.
6. Johnson-Kwochka A, Bond G, Becker, Deborah R, Drake RE, Greene MA.
Prevalence and quality of Individual Placement and Support (IPS) sup-
A182 ported employment in the United States. Adm Policy Ment Heal Ment
Understanding the critical elements of an Integrated Scaling up Heal Serv Res. 2017;44:311-319. doi:10.1007/s10488-016-0787-5.
Approach (ISA)
Marianne Farkas, Sigal Vax, Vasudha Gidugu, Kim Mueser, Chitra Khare,
Philippe Bloch
Boston University, Boston, MA, USA A183
Correspondence: Marianne Farkas ([email protected]) Applying implementation science for real world impact:
Implementation Science 2020, 15(Suppl 2):A182 operationalized core practice components, feasibility testing, and
next steps
Background William Aldridge, Rebecca Roppolo, Julie Austen, Robin Jenkins
To reduce the gap between research and practice in community University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
mental health services, there is a critical need to develop new Correspondence: William Aldridge ([email protected])
methods for scaling up evidence-based practices [1]. To increase Implementation Science 2020, 15(Suppl 2):A183
widespread access to effective practices, we are in the process of de-
veloping an Integrated Scaling Approach (ISA) for the efficient larges- Background
cale implementation of empirically based interventions for people Implementation science is at risk to suffer from the same challenge it
with psychiatric disabilities. was designed to address: a lack of translation into real world applica-
Materials and Methods tion. Complicating this challenge is that robust application of imple-
We conducted a scoping review of the literature across various fields, mentation science requires supporting behavior change at each of
including mental health, our own field of practice, as well as public individual, organizational, and system levels. “Technical assistance,”
health, business, and education, fields in which large scale imple- “facilitation,” and “implementation support” are terms often used to
mentation efforts are common. We then focused on examples of the describe the concept of “implementation practice.” Regardless of
implementation of employment initiatives, which have become an these labels, what drives the effective application of implementation
important focal point for change in practice and policies in mental science within real world environments?
health agencies and systems interested in the recovery of people Materials and Methods
with psychiatric disabilities [2-3]. While interventions have proven ef- Drawing from a review of relevant literature and our experience facili-
fective in supporting employment for this population, the number of tating the real-world application of implementation science, members
people benefiting from them remains limited [4-6]. The scoping re- of The Impact Center at FPG organized ten theoretically- and
view included interviews of a range of stakeholders, with experiences empirically-informed core practice components to strengthen imple-
in implementing such employment initiatives in their state or region mentation support processes. These ten proposed core practice com-
(whether successful or unsuccessful) and expertise in program lead- ponents underwent initial feasibility testing within two projects
ership, employment services, mental health and vocational rehabilita- involving implementation support for communities scaling an
tion services administration and policy. evidence-based system of parenting interventions. For more than two
Results years, implementation specialists have tracked their utilization of the
Both literature and interviews were analyzed to identify recurring practice components across interactions with community sites. Imple-
themes and critical components of large-scale implementation in mentation specialists working to build the capacity of intermediary
community mental health services. Analyses currently in process will partners also tracked their use of practice components. All community
identify characteristics of these critical components drawn from the and intermediary sites received monthly surveys to report process and
cross-field scoping review. A pilot test of the ISA and a final evalu- short-term outcomes. Intended long-term outcomes of implementation
ation will be conducted in two states in 2020-2021.The analyses of support (capacity to support implementation best practices at commu-
the scoping review and the results of the pilot and evaluation studies nity and intermediary levels) were assessed every six months.
will be used to create a handbook for intervention researchers, sys- Results
tem and program administrators and knowledge translation special- Initial results suggest that the ten proposed practice components are
ists to use, when scaling up new interventions in mental health. an effective way to organize the work of implementation support.
Conclusions Supported sites reported favorable process outcomes, such as ac-
In this presentation, we will share the data collection, analysis ceptability, feasibility, and appropriateness. Short-term outcomes,
process, and preliminary characteristics of the ISA as identified such as working alliance, have been useful markers for early suc-
through the comprehensive scoping review. cesses or challenges. Long-term capacity outcomes have demon-
strated improvement over time. Notwithstanding these strengths,
References Center implementation specialists voiced a need for greater clarity
1. Wiltsey-Stirman S, Gutner CA, Langdon K, Graham JR. Bridging the gap about operational activities related to the practice components. This
between research and practice in mental health service settings: An over- recently led members of Center to re-operationalize the components
view of developments in implementation theory and research. Behav to better support consistent application.
Ther. 2016;47:920-936. doi:10.1016/j.beth.2015.12.001.
Conclusions Participant expectations include: on-site attendance at presentations

Next steps include the development of a complete practice profile, by local and national D&I scholars, completion of readings between
stronger training and fidelity assessment materials, and formative sessions, and regularly working with an assigned mentor to generate
evaluation methods to test statistical associations between the com- a project to move their D&I effort ahead.
ponents and intended short- and long-term outcomes. Results
We started the program recruiting the capacity of 25 trainees. After
A184 our kickoff event, seven participants withdrew (n=3 perceived rele-
If you want more research based practice, you need more practice vance, n=2 distance/time, n=1 moved, n=1 unknown). We invited
based and early stage D&I trained researchers (borrowed and two trainees from the waitlist and now have 20 trainees who span
slightly changed from Larry Green) the translational spectrum: Basic research (n=5); Pre-clinical (n=4); Ef-
Rodger Kessler, Cady Berkel, Matthew Buman, Stephanie Brenhofern, ficacy/Adaptation (n=9); Implementation in clinical and community
Scott Leischow settings (n=12); Studying health outcomes at the population level
Arizona State University, Tempe, AZ, USA (n=7) and career stage [graduate student/postdoc (n=4); assistant
Correspondence: Rodger Kessler ([email protected]) professor (n=9); associate professor (n=5); clinical professor (n=1);
Implementation Science 2020, 15(Suppl 2):A184 and adjunct professor (n=1)]. Total n is greater than 20 because we
allowed trainees to select multiple areas. Each trainee was paired
Background with one of nine mentors.
While training programs in D&I research have emerged over the last Conclusions
few years, they have generally been limited to single institutions or We generated a multi institutional D&I training program. Recruitment
individuals traveling to national training sites. This has limited reach was easily accomplished. Loss of participants due to absence of dis-
of training and opportunities for multi-institutional development of tance learning needs attention and lack of fit to earlier stage transla-
broad D&I capacity. In addition, investigators must effectively engage tional scientists suggests we need to refine the material presented at
with other partners, from clinical trials centers to community partners the first event to better include those individuals. Little D&I training
and policymakers. Such engagement skills need to be embedded in has been developed for the basic end of the translational spectrum;
the core D&I training. This includes assisting earlier stage transla- we are attempting to fill that gap. We will report further on evalu-
tional scientists’ participation in next stage activities, with designing ation data and the projects that trainees generate.
for dissemination at the forefront.
We designed a yearlong D&I training program for Arizona State Uni- Publisher’s Note
versity faculty and other researchers across the state, borrowing from Springer Nature remains neutral with regard to jurisdictional claims
the TIDIRH curriculum. Implementation began in February 2019. in published maps and institutional affiliations.

Conferenc e Proceedings

Uploaded by

Copyright:

Available Formats

Conferenc e Proceedings

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Conferenc e Proceedings

Uploaded by

Copyright:

Available Formats

Implementation Science 2020, 15(Suppl 2):80

MEETING ABSTRACTS Open Access

Proceedings of the Fifth Biennial

wants their decisions to be supported by evidence, this information

Background Materials and Methods

Results levels of engagement. However, we also identified barriers (e.g., ensur-

Materials and Methods Materials and Methods

1) Increasing awareness of the needs, challenges and opportunities A48

Results strategies to enhance motivation whereas over-adapting adopters

Background implementation activity was observed, reflective of funding pres-

will be used to summarize the availability of psychometric informa- Results

on fidelity, and distractions/technical difficulties. Similarities and differ- References

implement the intervention in primary care settings. Results will be References

an Implementation Practice Core Competency Tool which will be fi- A106

Background adoption, implementation, and sustainability [2-4]. As part of a school-

A118 5. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the

A120 5. Eaton J, McCay L, Semrau M, Chatterjee S, Baingana F, Araya R, Ntulo C,

schools might be negatively influencing adoption. These results have Conclusions

8. Evans A, Ranjit N, Rutledge R, Medina J, Jennings R, Smiley A, Hoelscher A142

Background vaccination services for adolescents. To date, no study we are aware

Background are developing and implementing a two-phase screening study to

Background Materials and Methods

included lack of systematic alcohol screening procedures; variability Results

Conclusions Participant expectations include: on-site attendance at presentations

You might also like