Comparing Methods to Make Research More Patient Centered [Internet]

Background: Engaging patients and communities in research has emerged as a critical element to advancing health research, improving the relevance of research, and enhanced implementation of research findings. While methods of engaging patients exist, variability among methods and limited understanding of how engagement impacts research may limit broader use.

Objectives: The overall purpose of this Methods Award was to develop evidence on the impact of stakeholder engagement on research, which is a critical methodological gap identified by PCORI. Specifically we aimed to demonstrate whether input from patients and communities elicited using the Community Engagement Studio (CE Studio, formerly called Community Review Board) was more patient centered than input from a Translation Studio (T2 Studio), which is composed of researchers. To evaluate the input from studios, we developed a standard set of terms, Stakeholder Impacts in Research Taxonomy, to describe potential changes in research due to stakeholder input and a quantitative scale, which would be generally applicable to assessment and evaluation stakeholder engagement.

Methods: This methods study randomized 20 researchers to the CE Studio, a structured method of eliciting project-specific input from patients and community stakeholders, or a T2 Studio, which obtains expert advice from researchers. Any faculty member or research trainee at Vanderbilt University or Meharry Medical College was eligible to be randomized. Researchers who requested a CE Studio through StarBRITE, Vanderbilt's online portal for research support services, and agreed to be in the study were randomized to a CE Studio (a panel of community members or patients) or T2 Studio (a panel of researchers). Those who were randomized to the T2 Studio were also offered a CE Studio. Randomization to CE Studio or T2 Studio was done using random number generator software (www.r-project.org). Each studio panel was convened to provide project-specific input. The 153 stakeholders who participated as experts on CE Studios panels were patients, caregivers, or patient advocates identified by health status, health condition, or demographic variables based on the project-based needs of the 20 researchers randomized in this project. Stakeholders include individuals with diabetes, heart failure, Parkinson disease, sickle cell disease, and intensive care unit survivors. All stakeholder experts had experience as a partner or consultant on a research project or through serving on a research advisory board or committee. T2 Studio experts were researchers from Vanderbilt University or Meharry Medical College. All studios were recorded and transcribed, and experienced qualitative researchers analyzed the data. We compared the project-specific input elicited from CE Studios with that elicited from T2 Studios using qualitative analysis.

To develop the taxonomy (the Stakeholder Impacts in Research Taxonomy) describing and characterizing wherein the research process stakeholder input has the potential for impact and the types of changes, we used a 3-step approach involving patients in each step: (1) identified stakeholder impacts from existing literature and generated standard terms; (2) evaluated the terms using cognitive interviews and a panel of researchers and stakeholders; and (3) pilot tested the terms using qualitative analysis.

Because no such measure existed, we created and validated a quantitative instrument measuring patient (person) centeredness of research. We developed the Person-Centeredness of Research Scale (PCoR Scale) by (1) content and item generation, (2) item evaluation, (3) initial scale testing, (4) scale revision, and (5) revised scale testing. We tested the initial and revised scales by comparing ratings for abstracts from a PCORI conference with those for abstracts from a national translational research meeting. We determined the scale's internal consistency reliability with an exploratory factor analysis and Cronbach α computation. We also examined the interrater reliability of the final rating scale.

Results: The CE Studios generated input with more themes consistent with patient centeredness than the T2 Studios and also a striking number of patient experiences that were expressed in storytelling form. The Stakeholder Impacts in Research Taxonomy has 11 domains (representing categories of research activities) and 59 dimensions, flexibly framing the stakeholder engagement impact on translational research for increased understanding and more formal assessment. The domains are (1) preresearch, (2) infrastructure, (3) study design, (4) implementation, (5) analysis, (6) dissemination of research findings, (7) postresearch, (8) ethics, (9) process improvement, (10) engagement, and (11) communication. The PCoR Scale is a 7-item instrument using a 5-point Likert rating scale that successfully classified (based on expert panel assignment) as person centered 71.4% of the conference abstracts tested in discriminant analysis (mean total scores: 7.15 vs −2.08; t = 7.37; P < .001). An exploratory factor analysis showed the PCoR Scale to be unidimensional with an eigenvalue of 5.59 that explained 79.8% of the total variance, and we computed Cronbach α to be .95, showing high internal consistency reliability.

Conclusions: Patients and community members provided project-specific input across a broad range of research areas, and this input was more person centered than input from researchers. The Stakeholder Impacts in Research Taxonomy and the quantitative PCoR Scale can be used by others in the field to help standardize this work and evaluate the patient person centeredness of research products.

Limitations: Although the CE Studio can be used across many studies, it is a consultative method of engagement and may not elicit feedback reflecting the full range of stakeholder engagement. Because the intent of this study was to provide evidence to support the added value of stakeholder engagement, we compared input from stakeholders with input from researchers. We did not compare CE Studios with a different method of stakeholder engagement, which might result in more or less person centeredness.

Definitions: In this report, the term stakeholder includes patients, caregivers, patient advocates, and other community members, but not payers, policy makers, or health care product producers. Researchers are also stakeholders in research, by definition, but for the purposes of this report, the term stakeholder is used to distinguish someone who is a community representative stakeholder from others with interest in the research. We have used community/patient stakeholder where that clarification seemed appropriate. A community representative is a person whose primary affiliation is with a nonacademic, nonresearch community-based organization and/or who represents a defined community.

Note: The term patient centered, well-defined by PCOR, is being replaced by the broader term person centered by researchers in the field.^, This terminology shift occurred during the course of our project, and we use both terms, reflecting the terms used during the research and moving toward the newer term during development of the quantitative scale, as noted in the relevant methods section.