Extending the Calgary Audit and Feedback Framework into the virtual environment: a process evaluation and empiric evidence

Background: The Calgary Audit and Feedback Framework (CAFF) is a pragmatic, evidence-based approach for the design and implementation of in-person social learning interventions using Audit and Group Feedback (AGF). This report describes extension of CAFF into the virtual environment as part of a multifaceted intervention bundle to reduce redundant daily laboratory testing in hospitals. We evaluate the process of extending CAFF in the virtual environment and share resulting evidence of participant engagement with planning for practice change.

Methods: We describe an innovative virtually facilitated AGF intervention based on the CAFF. The AGF intervention was part of an intervention bundle which included individual physician laboratory test utilization reports and educational tools to reduce redundant daily laboratory testing in hospitals. We used data from recorded and transcribed virtual AGF sessions, post AGF session surveys and detailed field notes maintained by project team members. We used simple descriptive statistics for quantitative data and analyzed qualitative data according to the elements of CAFF.

Results: Eighty-three physicians participated over twelve virtual AGF sessions conducted across four tertiary care hospitals during the study period. We demonstrate that all prerequisite activities for CAFF (relationship building, question choice and data representation) were present in every virtual AGF session. Virtual facilitation was effective in supporting the transition of participants through different steps of CAFF in each session to lead to change talk and planning. All participants contributed to discussion during the AGF sessions. The post AGF session surveys were filled by 66% of participants (55/83), with over 90% of respondents reporting that the session helped them improve practice. 46% of participants (38/83) completed personal commitment to change forms at the end of the sessions.

Conclusions: Virtual AGF sessions, developed and implemented with fidelity to the CAFF approach, successfully engaged physicians in a group learning environment that led to change planning. Further studies are needed to determine the generalizability of our findings and to add to the literature on evidence-based virtual facilitation techniques.