Objective Assessment of Resident Teaching Competency Through a Longitudinal, Clinically Integrated, Resident-as-Teacher Curriculum

Objective: Competency-based training should be paired with objective assessments. To date, there has been limited objective assessment of resident-as-teacher curricula (RATC). We sought to assess the impact of a longitudinal RATC on postgraduate year-1 (PGY1) resident teaching competency using Observed Structured Teaching Encounters (OSTEs) for the skills of 1) brief didactic teaching [DT], 2) feedback [FB], and 3) precepting [PR].

Methods: A controlled, prospective, educational study was conducted from May 2015 to June 2016. The RATC consisted of a workshop series with reinforcement of key skills (DT, FB) during clinical rotations. Intervention residents participated in the RATC and completed OSTEs at the beginning and end of the academic year. A control group, PGY1 residents that matriculated the year previously, completed the OSTEs before starting their PGY2 year. OSTEs were reviewed by 2 blinded study personnel. We assessed reliability between raters via intraclass correlation coefficients and differences in OSTE scores via least squared mean differences (LSMD).

Results: In total, 92.5% (n = 37) of eligible control and 100% (n = 41) of eligible intervention residents participated. The OSTEs demonstrated excellent agreement between reviewers (DT: 0.99, FB: 0.89, PR: 0.98). A significant pre-post difference was demonstrated in the intervention group for DT (LSMD [95% confidence interval], 3.14 [2.49-3.79], P < .0001), FB (0.93 [0.49-1.37], P < .0001), and PR (0.64 [0.09-1.18], P < .022). A significant difference between the control and intervention groups was demonstrated for DT (3.00 [2.05-3.96], P < .0001).

Conclusions: Skill-based OSTEs can be used to detect changes in residents' teaching competency and may represent a potential component of programmatic evaluation of resident-as-teacher curricula.