Development of a Brief, Positively Framed Care Team Experience Measure

Background: Measures that quantify clinician fulfillment and engagement with work (clinician experience measures) have the potential to improve the quality, effectiveness, and enjoyment of patient care. Many existing measures of clinician fulfillment and engagement with work are relatively long and potentially burdensome, negatively framed, or address personal well-being. A measure with a small number of items that address positive attributes to which a team aspires may be a more useful measure to guide organizational improvement efforts.

Questions/purposes: In a series of cross-sectional studies we asked: (1) Can we develop a brief, positively framed, and team-oriented care team experience measure (CTEM)? (2) How does the CTEM perform relative to the Team Climate Inventory (TCI)?

Methods: In a first step, the clinical leadership group of a nascent multispecialty group practice curated 21 items, some from existing engagement and burnout measures and others based on suggestions from clinical leaders. After pilot testing and interviews with specific clinicians, these items were then administered to all clinicians in the practice, and 44% (120 of 274) participated. Factor analysis identified three groups of items (factors). In a second step, 43% (493 of 1135) of patient-facing employees (75% [368] of whom were women and 45% [222] of whom were ages 42 to 61 years) of a statewide musculoskeletal specialty practice rated 12 items (two from each of the factors identified in the first stage and six new items recommended by administrative leaders) and completed the TCI. Participants were then randomly allocated to a learning cohort (70% [343 participants]), in which we performed factor analysis and item response theory to develop a new CTEM and validation cohort (30% [150 participants]) to test the consistency of the findings. The best-performing items in terms of factor loadings (the degree to which an individual item represents an identified factor or "theme"; we chose a minimum of 0.4 as items above this threshold are generally regarded as stable), item discrimination (ability to measure different experiences among participants), item difficulty (the ability of an item to contribute to variation in scores), and model fit (the degree to which the estimates of the statistical model align with the observed data) were selected to form a new three-item CTEM. The CTEM was subsequently evaluated for internal consistency (Cronbach alpha, which measures the extent to which different subsets of the included items would provide the same measurement), floor and ceiling effects (the percentage of the lowest and highest possible scores, with high percentages indicating that information is lost because of the inability of a tool to measure the extremes of variation), and correlation with the TCI.

Results: Factor analysis identified two groups of items representing "effectiveness" (nine items, including "I am proud of the work we do") and "collaboration" (three items, including "Our team encourages everyone's input before making changes"). The best-performing items of "effectiveness" (two items) and "collaboration" (one item) were selected to form the three-item CTEM, which had good internal consistency (Cronbach α = 0.77; > 0.7 generally considered acceptable); a notable ceiling (37%; higher than desired, common with experience measures, and-in our opinion-acceptable for routine use given the trade-off of limited participant burden) and limited floor effect (1.3%); strong correlation with the TCI (Spearman ρ 0.77 [95% confidence interval 0.70 to 0.85]), which supports convergent validity (part of construct validity, meaning correlation with measures of related concepts); and similar correlations with participant characteristics compared with the TCI, supporting construct and criterion validity (association with real-world outcomes).

Conclusion: A small number of positively framed, team-oriented questions (the CTEM) can quantify clinician and care team experience and provide a practical and feasible method for health organizations to monitor and improve the experience of their clinicians.

Clinical relevance: Routine measurement of organizational climate at regular intervals using a brief, user-friendly, team-oriented measure might help organizations improve their culture and help their clinicians feel appreciated and engaged.