Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS)

Katharina Szota; Hanna Christiansen; Gregory A Aarons; Mark G Ehrhart; Anne Fischer; Rita Rosner; Regina Steil; Antonia Barke

doi:10.1038/s41598-023-32282-4

Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS)

Sci Rep. 2023 Mar 31;13(1):5311. doi: 10.1038/s41598-023-32282-4.

Authors

Katharina Szota¹, Hanna Christiansen², Gregory A Aarons³, Mark G Ehrhart⁴, Anne Fischer⁵, Rita Rosner⁶, Regina Steil⁵, Antonia Barke⁷

Affiliations

¹ Department of Psychology, Philipps-University of Marburg, Gutenbergstr. 18, 35032, Marburg, Germany. [email protected].
² Department of Psychology, Philipps-University of Marburg, Gutenbergstr. 18, 35032, Marburg, Germany.
³ Department of Psychiatry, University of California, 9500 Gilman Drive (0812), La Jolla, San Diego, CA, 92093-0812, USA.
⁴ Department of Psychology, University of Central Florida, P.O. Box 161390, Orlando, FL, 32816-1390, USA.
⁵ Department of Psychology, Goethe University Frankfurt, Varrentrappstr. 40-42, 60486, Frankfurt am Main, Germany.
⁶ Department of Clinical and Biological Psychology, Catholic University of Eichstätt-Ingolstadt, Levelingstr. 7, 85049, Ingolstadt, Germany.
⁷ Division of Clinical Psychology and Psychological Intervention, Institute of Psychology, University of Duisburg-Essen, Universitätsstr. 2, S06 S03 B24, 45141, Essen, Germany.

Abstract

Organizational implementation climate is an important construct in implementation research to describe to what extent implementation is expected, supported, and rewarded. Efforts in bridging the research-practice gap by implementing evidence-based practice (EBP) can benefit from consideration of implementation climate. The Implementation Climate Scale (ICS) is a psychometrically strong measure assessing employees' perceptions of the implementation climate. The present cross-sectional study aimed at providing a German translation and investigating its psychometric properties. The translation followed standard procedures for adapting psychometric instruments. German psychotherapists (N = 425) recruited online completed the ICS, the Evidence Based Practice Attitudes Scale (EBPAS-36D) and the Intention Scale for Providers (ISP). We conducted standard item and reliability analyses. Factorial validity was assessed by comparing an independent cluster model of Confirmatory Factorial Analysis (ICM-CFA), a Bifactor CFA, a Second-order CFA and an (Bifactor) Exploratory Structural Equation Model (ESEM). Measurement invariance was tested using multiple-group CFA and ESEM, convergent validity with correlation analysis between the ICS and the ISP subjective norms subscale (ISP-D-SN). The mean item difficulty was p_i= .47, mean inter-item correlation r = .34, and mean item-total correlation r_itc = .55. The total scale (ω = 0.91) and the subscales (ω = .79-.92) showed acceptable to high internal consistencies. The model fit indices were comparable and acceptable (Second-order CFA: RMSEA [90% CI] = .077 [.069; .085], SRMR = .078, CFI = .93). Multiple-group CFA and ESEM indicated scalar measurement invariance across gender and presence of a psychotherapy license. Psychotherapists in training reported higher educational support for EBP than licensed psychotherapists (T = 2.09, p = .037, d = 0.25). The expected high correlation between the ICS and the ISP-D-SN was found (r = .59, p < .001). Results for the German ICS confirm good psychometric properties including validity.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Cross-Sectional Studies
Evidence-Based Practice* / methods
Factor Analysis, Statistical
Germany
Humans
Mental Health*
Psychometrics
Reproducibility of Results
Surveys and Questionnaires