Objectives: (1) To compare staff members' opinions of long-term care (LTC) residents' ability to complete a customer satisfaction survey (CSS) with a measure of cognition (MDS-COGS) derived from Minimum Data Set data; and (2) to examine the association between CSS answer reliability and MDS-COGS score.
Study design: Retrospective comparison of the staff's assessment and MDS-COGS score for each respondent, as well as a prospective comparison of MDS-COGS scores with reliability measures from repeated survey administration.
Setting: A 100-bed Veterans Affairs (VA) nursing home.
Methods: We administered a CSS designed by our VA network following an assigned protocol. We later calculated each respondent's MDS-COGS score (grouped into 4 categories) and compared it with the staff's opinion of whether that resident was "capable of responding" (yes/no) to a CSS. We subsequently modified the CSS for low reading level and low vision, and randomly selected 40 LTC residents for repeated survey administration (T1 and T2 1 week later). Test-retest reliability was estimated by examining the extent to which T1 and T2 answers agreed (agreed exactly; meaningfully agreed as defined by VA network personnel who designed the survey; or meaningfully agreed as decided by paper authors).
Results: Staff judged that 25 of 76 LTC residents were not and 51 of 76 were capable of responding to the CSS. In 82% of cases, MDS-COGS score category and staff opinion agreed ("no cognitive impairment"/"mild-moderate cognitive impairment" with "able to complete"; and "moderate-severe cognitive impairment"/"severe cognitive impairment" with "unable to complete"). Cohen's kappa was 0.57 with a P value of <0.001. Of the 40 LTC residents randomly selected for repeated survey administration, 32 successfully completed surveys at T1 and T2. Higher MDS-COGS scores, suggesting greater cognitive impairment, were significantly associated with lower answer reliability. The answers given by LTC residents changed meaningfully (by network criteria) from T1 to T2 by 12%, 27%, and 28% across categories of no-to-mild cognitive impairment, mild-to-moderate impairment, and moderate-to-severe impairment, respectively. The rate of decline in reliability, however, was small for each unit of increase in MDS-COGS.
Conclusions: Although the staff's opinions about residents' ability to complete a CSS were significantly associated with an objective MDS-COGS assessment, staff nonetheless failed on several occasions to let cognitively intact residents complete their own surveys. Although higher MDS-COGS scores were significantly related to lower reliability, there were no natural cut-points in the data to suggest a participate/do not participate line and the trend line between MDS-COGS level, and percent of questions answered reliably was not very strong. These results suggest that LTC residents with mild to moderate dementia might be capable of reliably answering certain types of customer satisfaction surveys.