Simulation-based objective assessment discerns clinical proficiency in central line placement: a construct validation

Chest. 2010 May;137(5):1050-6. doi: 10.1378/chest.09-1451. Epub 2010 Jan 8.

Abstract

Background: Central venous catheterization (CVC) is associated with patient risks known to be inversely related to clinician experience. We developed and evaluated a performance assessment tool for use in a simulation-based central line workshop. We hypothesized that instrument scores would discriminate between less experienced and more experienced clinicians.

Methods: Participants included trainees enrolled in an institutionally mandated CVC workshop and a convenience sample of faculty attending physicians. The workshop integrated several experiential learning techniques, including practice on cadavers and part-task trainers. A group of clinical and education experts developed a 15-point CVC Proficiency Scale using national and institutional guidelines. After the workshop, participants completed a certification exercise in which they independently performed a CVC in a part-task trainer. Two authors reviewed videotapes of the certification exercise to rate performance using the CVC Proficiency Scale. Participants were grouped by self-reported CVC experience.

Results: One hundred and five participants (92 trainees and 13 attending physicians) participated. Interrater reliability on a subset of 40 videos was 0.71, and Cronbach a was 0.81. The CVC Proficiency Scale Composite score varied significantly by experience: mean of 85%, median of 87% (range 47%-100%) for low experience (0-1 CVCs in the last 2 years, n = 27); mean of 88%, median of 87% (range 60%-100%) for moderate experience (2-49 CVCs, n = 62); and mean of 94%, median of 93% (range 73%-100%) for high experience (> 49 CVCs, n = 16) (P = .02, comparing low and high experience).

Conclusions: Evidence from multiple sources, including appropriate content, high interrater and internal consistency reliability, and confirmation of hypothesized relations to other variables, supports the validity of using scores from this 15-item scale for assessing trainee proficiency following a central line workshop.

Publication types

  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Catheterization, Central Venous / methods*
  • Certification
  • Clinical Competence / standards*
  • Guidelines as Topic
  • Humans
  • Manikins*
  • Medical Staff / education*
  • Prospective Studies
  • Pulmonary Medicine / education
  • Pulmonary Medicine / instrumentation
  • Reproducibility of Results