Development and Validation of a Geriatrics Knowledge Test to Evaluate Geriatrics Fellowship Programs

Background/objectives: Although the Accreditation Council for Graduate Medical Education requires that geriatrics fellowship programs be evaluated, evaluation is challenging because of lack of appropriate instruments. The purpose of this study was to develop and validate a geriatrics knowledge test appropriate for evaluation of geriatrics fellowship programs.

Design: Initial and replication cross-sectional validation studies.

Setting: Academic medical center.

Participants: Initial study: 11 postgraduate year (PGY)-2 and five PGY-3 internal medicine residents, eight incoming and eight graduating geriatrics fellows, and 11 geriatrics faculty (N = 43). Replication study: nine graduating fellows and three mid-year fellow cohorts (n = 11, 8, and 9) (N = 37).

Measurements: A geriatrics knowledge test was developed consisting of 31 multiple-choice questions (MCQs) selected from a 54-item pool of MCQs that the authors created. Selection criteria included content appropriateness, item correlation with total score, item discriminatory power, and item difficulty.

Results: The instrument demonstrated high reliability (Cronbach alpha = 0.83) and known group validity. The mean percentage correct scores on the knowledge test were progressively higher with more geriatrics training (P < .001). The replication study continued to show patterns of progressive increases in score with additional training. There were no floor or ceiling effects.

Conclusion: A 31-item geriatrics knowledge test demonstrates sound reliability and validity. The findings support that it is appropriate as a tool for evaluation of geriatrics fellowship programs.