Curriculum Using the In-Situ Operating Room Setting

Objective: The American College of Surgeons/Association of Program Directors in Surgery is a comprehensive, simulation-based curriculum for General Surgery residents which exists in 3 phases. While phases 1 and 2 deal with core skills and advanced procedures respectively, phase 3 targets team-based skills. To date, the 3rd phase of this curriculum has not seen wide scale implementation. This is a pilot study to verify the feasibility of implementing the phase 3 curriculum in the in-situ setting.

Design: In our initial attempt to implement Phase 3 at our institution, we chose to perform the training in an in-situ setting within an operating room (OR) at our main hospital, despite our having a separate simulation center. By choosing the in-situ OR environment for this training we were able to minimize concerns regarding resident and faculty availability and able to successfully complete 8 separate sessions during the academic year. During 7 sessions, 2 separate scenarios were performed while a single scenario was performed in 1 session. This single session was excluded from analysis, leaving a total of 14 scenarios to evaluate. The unique scenarios included laparoscopic crisis, postoperative myocardial infarction, anaphylaxis, and postoperative hypotension. All sessions were audiovisually recorded. In order to evaluate the effect of the training, the videos were viewed by 3 independent reviewers and all surgery, anesthesia and nursing participants were rated using the NOTECHs II scale. Degree of inter-rater agreement was established. The difference between the first and second simulations on the same day was then assessed. In addition, participant opinions of the simulations were assessed through electronic surveys following the training.

Setting: Tertiary Care University Hospital.

Participants: We performed a total of 8 sessions, for a total of 15 scenarios. Eight surgery residents at the postgraduate year 1 (PGY1)-PGY3 level, 16 anesthesia residents at the PGY3-PGY4 level, 16 nurses and 13 ancillary staff participated.

Results: From the first to the second scenario, the total team NOTECHs II score increased from 69.4 ± 1.4 to 77.3 ± 0.5 (p = 0.007). The NOTECHs II scores for each subteam also improved, from 24.2 ± 0.6 to 26.4 ± 0.5 (p = 0.007) for surgery residents, 23.7 ± 0.9 to 26.7 ± 0.4 (p = 0.03) for anesthesia, and 21.6 ± 0.3 to 24.3 ± 0.5 (p = 0.01) for nursing. The inter-rater reliability as measured by Kendall's coefficient of concordance was modest for the whole team score. Most of the participant responses were either favorable or strongly favorable.

Conclusion: The in-situ OR environment is both a unique and effective setting to perform team-based training. Furthermore, training in the in-situ setting minimizes or removes many of the logistic issues involved in designing and implementing team-based training curricula for general surgery residency programs. However, we found that administrative and departmental (surgery, anesthesia, and nursing) "buy in" as well as protected faculty time for education were all necessary for in-situ training to be successful. NOTECHs II is an established scale for the evaluation of teams in this simulation setting and appears to be a valid tool based on the results of this study. However, further assessment of inter-rater reliability as well as improved training of evaluators are necessary to determine if inter-rater reliability can improve.