The SAFE-T upper endoscopy tool: a web-based application for the point-of-care evaluation of gastroenterology fellow performance in upper endoscopy

Navin L Kumar; Guillaume Kugener; Kelly E Hathorn; Molly L Perencevich; Kunal Jajoo; John R Saltzman

doi:10.1093/gastro/goaa031

The SAFE-T upper endoscopy tool: a web-based application for the point-of-care evaluation of gastroenterology fellow performance in upper endoscopy

Gastroenterol Rep (Oxf). 2020 Jul 29;8(6):431-436. doi: 10.1093/gastro/goaa031. eCollection 2020 Dec.

Authors

Navin L Kumar^{1

2}, Guillaume Kugener³, Kelly E Hathorn^{1

2}, Molly L Perencevich^{1

2}, Kunal Jajoo^{1

2}, John R Saltzman^{1

2}

Affiliations

¹ Division of Gastroenterology, Hepatology and Endoscopy, Brigham and Women's Hospital, Boston, MA, USA.
² Harvard Medical School, Boston, MA, USA.
³ Massachusetts Institute of Technology, Cambridge, MA, USA.

Abstract

Background: Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise, web-based assessment tool to evaluate real-time fellow performance in upper endoscopy.

Methods: We developed the Skill Assessment in Fellow Endoscopy Training (SAFE-T) upper endoscopy tool to capture both summative and formative feedback in a concise, five-part questionnaire. The tool mirrors the previously validated SAFE-T colonoscopy tool and is administered electronically via a web-based application. We evaluated the tool in a prospective study of 15 gastroenterology fellows (5 fellows each from Years 1-3 of training) over the 2018-2019 academic year. An independent reviewer evaluated a subset of these procedures and completed both the SAFE-T and Assessment of Competency in Endoscopy (ACE) upper endoscopy forms for reliability testing.

Results: Twenty faculty completed 413 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score differentiated each sequential fellow year of training, with first-year cases having lower performance than second-year cases (3.31 vs 4.25, P < 0.001) and second-year cases having lower performance than third-year cases (4.25 vs 4.56, P < 0.001). The mean SAFE-T overall score decreased with increasing case-complexity score, with straightforward compared with average cases (3.98 vs 3.39, P < 0.001) and average compared with challenging cases (3.39 vs 2.84, P = 0.042). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a Kappa agreement statistic of 0.815 (P = 0.001). The SAFE-T overall score also highly correlated with the ACE upper endoscopy overall hands-on score (r = 0.76, P = 0.011).

Conclusions: We developed and validated the SAFE-T upper endoscopy tool-a concise and web-based means of assessing real-time gastroenterology fellow performance in upper endoscopy.

Keywords: assessment; education; endoscopy; fellowship; tool; trainee.