Actionable adverse event monitoring and feedback to improve thoracic surgical care

Introduction

There is a great need for a learning health care system that continuously and systematically utilizes data to inform improvements in care. Surgeons have traditionally fostered a culture of independence, innovation and individual skill acquisition. Surgical excellence and improved patient outcomes rely less so on individual surgeon proficiency, but increasingly relate to multidisciplinary team efficiency, workflow dynamics and collaborative network-based care. While reasons for postoperative adverse events (AEs) often appear to lie with the sole-surgeon performing the operation, they are, in fact, related to both individual and system-wide cognitive, patient-centric and team factors. Given the critical importance of AEs to both patients and healthcare providers, there is a vital need to identify data-driven methods to reduce their occurrence. Due to the nature of thoracic surgery and the inability to eliminate uncertainty, AEs can never be eliminated completely; however, they can be significantly reduced, and thoracic surgeons are globally united in their relentless pursuit of reducing AEs and improving care. Innovative and supportive environments are required to fulfill such a common goal, including identifying ways to predict errors, foster and support best practices, while all along not demeaning less successful surgical groups, individual surgeons or other members of the multidisciplinary team. In this narrative review, we will describe the various approaches to monitoring AEs, and how to provide actionable feedback to those reducing or managing AEs, ultimately demonstrating potential avenues to improve patient care.

Importance of post-operative AEs

Postoperative AEs (defined as a deviation from expected recovery after surgery) are prevalent in all surgical disciplines. Expected recovery implies a healthy ‘textbook’ recovery, requiring no additional monitoring or therapy that is otherwise required at a baseline. Because of its complex and anatomically vital structures, and potential perturbation to vital physiology involving the thorax and chest wall mechanics, thoracic surgeries produce AEs at a higher frequency than many other classes of surgery (1-3). In addition to the technical challenges, the thoracic surgery patient population is often older, living with frailty as well as concomitant cardiopulmonary and/or multiple comorbidities. Although distinct clinical entities, frailty may be exacerbated by concomitant comorbidities, with the former representing a multidimensional syndrome characterized by a diminished physiological and psychological capacity to respond to operative stresses and, thus, maintain homeostasis (4).

Hospital mortality following thoracic surgery has dramatically decreased over the past several decades (5). The reasons for this improvement are multifactorial, and advances in surgical technique, patient selection, and perioperative care practices are some of the most pertinent ones. However, the rate of AEs remains significant, occurring in 30–60% of cases depending on the type of major thoracic surgery (6). The most common thoracic AEs include prolonged alveolar air leak (PAAL), postoperative pulmonary complications (POPC), atrial fibrillation (AFIB), and anastomotic leak (AL). Due to their frequency and prevalence, thoracic surgeons must be both familiar with, and have mitigating plans for, all postoperative AEs, which must be managed expeditiously when they occur.

Despite our general improvement of the care of thoracic surgery patients, the harmful effects of AEs are far-reaching and include both physical and psychological consequences. AEs are associated with an increased risk of mortality (7-9), impaired postoperative recovery (10), increased readmission rates (11), and (in the case of malignancy) increased incidence of oncological recurrence with worsened oncological outcomes (12). All AEs, regardless of severity, are associated with prolonged hospital length of stay (LOS) (13). Poor patient experiences are also subsequently reported with a simultaneous loss of trust in the healthcare system (14,15). These are all patient-centered harms caused by post-operative AEs, which translate into greater costs for hospitals.

Costs of AEs

Surgical AEs have substantial impacts on healthcare costs (16). In addition to the initial cost of a surgical procedure, the economic implications of AEs are staggering (7,17,18). While some AEs are inevitable, it has been estimated that between 37–51% of reported AEs are potentially preventable and cost the Canadian hospital system $397 million/year (19,20). Such estimates are based on increased LOS, intensive care unit (ICU) admission, reoperation, and 30-day readmission. These costs have been demonstrated to be increasing as the thoracic surgery population ages with higher rates of frailty, higher risk of complications, and greater risk of long term disability (21,22).

Despite economic pressures placed on overburdened health care systems worldwide, a rigorous evaluation of the economic impact of thoracic surgical AEs was lacking. As such, a systematic review of published literature was performed, highlighting that literature estimates are available for hospital costs for common AE, and are as follows: AL (USD$49,278) and pneumonia ($12,258) following esophagectomy, and prolonged air leak ($2,556), respiratory failure ($19,062), empyema ($30,189), pneumonia ($15,362), recurrent laryngeal nerve injury ($16,420) and arrhythmia ($6,835) following lobectomy (23). This study involved an iterative approach, using the Canadian Agency for Drugs and Technologies in Health (CADTH) to identify relevant economic studies. Methodological quality was appraised using the Critical Appraisal Skills Program (CASP) checklist. The results found a wide range of pooled mean costs of complication that reflect the heterogeneity in both costs of specific AEs and definitions of AEs. In addition, the review does not include any quantification of the broader health system and societal costs related to outpatient and out of pocket costs associated with AEs, as there is no published data.

A better understanding of the hospital costs of AEs are required to support quality improvement (QI) efforts aimed at reducing the highest incidence and most expensive AEs. Despite important efforts made by QI initiatives to reduce AEs, QI must also focus on improving the overall value of care provided. Here, an economic assessment of the most common and costly AEs can be integral in informing high-value interventions that carry a cost of their own, allowing a more nuanced and targeted approach to QI efforts across Canadian thoracic surgery centres. By reducing ‘low-value’ care and targeting high-value inexpensive interventions that reduce costly AEs, a positive synergistic effect of improving health outcomes for Canadian patients and reducing inefficiencies in healthcare delivery would be achieved. Such benefits would be seen for patients (improved outcomes), providers (improved work satisfaction), hospitals (reduced healthcare costs) and society at large (healthier population, reduced costs).

Variation in practice

Concomitant to reducing value-based QI to reduce harmful costly AEs, standardization of care is a collaborative objective. A 2015 report highlighted unacceptably high practice variation in how Canadian provinces deliver cancer care services, with a 10% difference in 5-year cancer survival rates among provinces, highlighting the need to also standardize practice and patient outcomes (24). Other studies have reported similar variation in practice, finding that the lack of infrastructure for pan-Canadian data collection and reporting is a primary contributor to the challenge of identifying best practices in surgical care across the country (25-27).

Esophageal cancer, a leading cause of cancer-related death (28), is managed by thoracic surgeons within a multidisciplinary team in regionalized cancer centres. However, despite the regionalization of esophageal cancer care in Canada, there remains significant regional variability in health care usage during diagnostic workup and subsequent treatment (29). A goal of regionalization is to improve patient-outcomes by balancing referrals to high-volume centres (in Canada, equivalent to 20 esophagectomies/year) with the potential travel burden experienced by patients and their families (30,31). High-volume centres may offer better management and expedited treatment of surgical AEs, which has been documented for high-risk surgical procedures, including esophagectomies (32).

Standardization of care and centralization of thoracic oncology has become a cornerstone of patient-centered care, with evidence demonstrating that various practice variations are linked to inconsistent, and potentially worse, patient outcomes, particularly in the frail (33,34). Assessing a patients’ individual risk of perioperative complications has been the subject of much inquiry. These risks encompass many components including surgical factors (e.g., technique, anastomotic factors, and blood flow factors) as well as patient factors (e.g., age, frailty, co-morbidities, and commensal microbiology). In summary, both the need to improve and standardize care require the capacity to measure AEs.

AEs classification

Among the chief issues in AE management has been the question of establishing a standardized system by which to classify the type and severity of AEs. The development of such a standardized classification system forms the basis for continuously monitoring AEs as it allows for uniformity of reporting, comparison across time, and comparison across centers. Over the past three decades, there has been an increased attention towards AE assessment, resulting in the formation of a number of classification systems such as the Clavien-Dindo classification along with a recent system developed by our group to classify thoracic AEs that is discussed further below.

The Clavien-Dindo classification was developed in 2004 as a revision of an earlier classification first proposed in 1992 (35). In this system, AEs are graded based on their severity, which is proportional to the level of therapy needed to treat the complication. It consists of five severity grades, where higher grades correspond to increasingly exhaustive therapies being employed, with the highest grade (Grade V) corresponding to patient mortality (Table 1). Additionally, two of the five groups are each divided into two subgroups. Grade III, which is any AE that requires surgical, endoscopic, or radiological intervention, is divided into IIIa (intervention not under general anesthesia) and IIIb (intervention under general anesthesia). Grade IV, which is any life-threatening complication requiring ICU management, is divided into IVa (single organ dysfunction) and IVb (multiorgan dysfunction). Finally, a suffix “d” may be used with any grade to indicate the presence of a disability, defined as any impairment in body function (Table 1). Since its first inception in 1992, followed by its revision in 2004, this classification system has been employed across several surgical specialties and validated in a large and diverse cohort of patients (36-39). The Clavien-Dindo AE classification system has stood the test of time, providing a simple yet adequately complex method to measure AE incidence and severity reliably and reproducibly.

Other surgical quality systems do not collect either severity of an AE, and some do not collect many AEs that occur specifically after thoracic surgery. Due to this limitation, in 2010, a classification system known as the Thoracic Morbidity and Mortality (TM&M) system was developed based on the Clavien-Diondo classification. The TM&M was an attempt to create a standardized system for classifying all postoperative AEs following thoracic surgery that would help better inform surgeons regarding Morbidity and Mortality (M&M) rates (3). Where appropriate, the grade of a specific complication was modified to more closely reflect AEs experienced by thoracic surgical patients (Table 2). Additionally, Grades I and II are designated as minor complications while Grades III and IV are considered major. Grade V corresponds to patient mortality. All grades are significant from a patient perspective, as all AEs are associated with increased risk of prolonged LOS, with lower grade AEs (Grades I and II) a lower odds ratio of prolonged LOS compared to patients with higher grade AEs (Grades III and IV) (3). To evaluate the reliability and reproducibility of the TM&M system, a 31-item questionnaire revealed excellent reliability and reproducibility of TM&M system (40), with subsequent national and international adoption (41-43).

AE harmonization & measurement

Due to the existence of several AE classification systems associated with differing national and international surgical associations, a problem that emerges is how one can translate and integrate AE data across multiple classifications and, by extension, various institutions both regionally and internationally. This problem poses a barrier to standardizing and evolving surgical quality beyond the limitations of surgeon-based and regional variation in care. In 2021, a method was developed to address this limitation via the creation of a system which enables universal data entry into separate international databases (44). This system consists of a software-friendly set of four drop-down menus (three single select: system, AE, grade; and one multiple selections: AE qualifier) that enables the inputted AE to be harmonized and classified across all major thoracic international associations, including the Canadian Association of Thoracic Surgeons (CATS), the European Association of Thoracic Surgeons (ESTS), the Society of Thoracic Surgeons (STS), the Esophagectomy Complications Consensus Group (ECCG), and the National Surgical Quality Improvement Program (NSQIP). The development of this system was intended to assist the harmonization of AE data collection across the mentioned associations and their classification systems, thereby providing an avenue for the emergence of coordinated data collection and QI networks (44,45).

As AEs occur in a dynamic and complex fashion, it is necessary for surgeons and their teams to prospectively measure and classify AEs, ideally reviewing all new AEs on a weekly basis, with collegial discussion sometimes required to adjudicate challenging AEs. For example, adjudicating whether respiratory failure is due to pneumonia, aspiration or something else may require expertise and deliberation. Although a retrospective chart-based review by trained personnel of well-defined AEs is valid and reliable, our experience is that if the goal is to capture all AEs and their severity, prospective AE collection involving a thoracic surgical team of staff surgeons, residents, and allied care staff if appropriate, offers the most valid AE measurement approach. Regardless, a vital aim and motivation of AE collection is to enable subsequent actionable feedback to improve care.

AEs feedback

Actionable morbidity & mortality rounds

M&M rounds find their roots in the early 20^th century, where the American surgeon Ernest Armory Codman, fastidiously and systematically tracked the outcomes of his patients, allowing for comparison amongst his peers. This practice helped initiate the beginnings of surgeon-led QIs, and importantly, fostered a culture of humility and transparency, and conceived of the potential benefits of standardized patient pathways. At its heart, M&M rounds aim to decrease medical errors and patient related AE (46). However, arguments against traditional M&M rounds suggest that the practice is limited by a myopic focus on errors and harm and have often failed to reduce preventable medical errors (47). Consequently, the concept of a structured actionable M&M rounds model through the lens of an innovative educational intervention has since been brought forward (Figure 1) (48). Here, the educational intervention focused on patient safety and care through an interprofessional M&M rounds (Figure 2). Participants would be able to (I) describe the importance of M&M as an opportunity to improve care; (II) select appropriate and actionable M&M cases; (III) analyze an M&M case to identify relevant cognitive and system issues; and (IV) generate specific, concrete actionable recommendations to address the identified issues, to be subsequently discussed (for action or not) at Divisional Rounds. This model was initially tested in emergency medicine and trauma care, but is applied in other surgical specialties (i.e., general surgery, thoracic surgery). Efforts to reduce or limit complications following surgical intervention are increasingly important and are being taught in medical education as a groundwork for future humble, and responsive surgeons and physicians (49,50). While it is impossible to eliminate all AEs following surgery, the ability to learn from such important patient outcomes will enhance insights into their root cause, the system level issues and ultimately improve the multidisciplinary team’s agency. At the individual surgeon level, actionable M&M rounds allows identification of cognitive issues that can translate into improved clinical guidelines and practice.

Figure 1 The Ottawa M&M model. Adapted from (48) with permission. M&M, Morbidity and Mortality.

Figure 2 Positive deviance seminar.

PD

As surgical AEs are a critical determinant of surgical quality, and technical proficiency undoubtably contributes to optimal patient outcomes, surgical excellence and improved outcomes have been increasingly shown to be superior when an integrated team is associated with large numbers of cases and experience (51). To further identify data-driven methods to reduce AE occurrence, the concept of PD has emerged in healthcare literature. PD is a relatively recent approach to QI built on the observation that in any community of individuals (including surgeons), there exist members whose uncommon practices provide them with asymmetrically better outcomes compared to the rest of the group. These individuals, referred to as those experiencing “positive deviance” or “positive outliers”, are identified and their practices are highlighted and shared among their peers, after which these practices are adopted by the group with the expectation of community-wide improvement of outcomes. In the context of healthcare, this is a process designed to distinguish exemplary practice performed by individuals, teams, or organizations, whose actions lead to improved safety and quality of given care (52).

PD was first suggested in the 1970s and later successfully implemented in the 1990s as part of a pilot project to improve child malnutrition in Vietnamese villages (53-56). The use of PD has since been further investigated and successfully used in a wide array of domains outside of nutrition, including education (57-60), agriculture (61,62), and healthcare. Within the latter, PD has been used to reduce medication errors in tertiary care hospitals (63), improve hand hygiene compliance (52), and reduce the incidence of healthcare-associated infections (52,64). Its wide-ranging success as a QI tool rests both in the observation that there exist best performers in every setting as well as in its scalability according to the resources available in the context in which it is administered. Given that practice that demonstrates PD commonly have access to the same resources as their peers, their superior performance is often the result of a specific academic focus, increased experience or focus in that domain of practice, involving specific pre-, intra and post-operative choices and practices rather than favorable circumstances.

Based on published recommendations (42) and due to the sensitive nature of AEs, a novel and efficient use of the PD approach was developed and called ‘PD seminars’. Here, the PD seminar involved the following: Self-Assessment, review of Best Evidence (i.e., literature review), then data-informed best Experience (i.e., PD), that culminates in actionable consensus Recommendations (SABEER). The process is as follows: (I) An AE or outcome and procedure is chosen by the target group for the subsequent process (e.g., air leak after lobectomy, AL after esophagectomy, or LOS after pneumonectomy). (II) For self-assessment, surgeons see their current performance to a range of possible useful comparators (e.g., anonymized peer colleagues, past performance, division averages, or the full census of thoracic surgical patients). Self-assessment data can be subdivided based on disease, date range, procedure and AE. (III) Surgeons then meet (in person or virtually) where a review of pertinent literature is provided by a member, summarizing key clinical trials, guidelines, and existing recommendations relevant to the discussion. (IV) Data is then summarized anonymously for all surgeons (e.g., surgeon A, B, C, ...). The group identifies the best performer(s) (may be more than one) who demonstrate PD (i.e., best results) with respect to the outcome and procedure being discussed (rarely, there is no positive outlier, and the group moves on to step 5). The surgeon(s) demonstrating PD tell their “story”, why they think their practice leads to superior outcomes. (V) Last, a collegial discussion of strategies used by PD surgeon(s) and findings from the literature review collectively result in the generation of actionable consensus recommendations, which the audience rates their acceptance of each intervention immediately and simultaneously with an audience polling app (e.g., Mentimeter). At the conclusion of the seminar, all participants commit to adopting the actionable recommendations, effective immediately. As the SABEER PD seminar is a repeatable process, each time focused on a specific AE and surgical procedure.

Initial experience with this approach has been favourable. The above SABEER PD seminar was performed 19 times in 7 Canadian thoracic surgery centers, each time focused on a procedure specific AE, yielding 132 consensus recommendations. An evaluation was first completed in a single center implementation (03/2013–02/2016), where a 34% reduction in AFIB, a 38% reduction in air leak, and a 25% reduction in AL after major thoracic operations was observed in a longitudinal pre-post design (65). Surgeon interviews highlighted favourable impressions of the team-building, positive and supportive nature, and patient-centered process (66). More recently, multicentre SABEER PD seminars were completed focussing on AEs and LOS following pulmonary resection (67) and AL and LOS following esophagectomy (68). Ultimately, SABEER provides positive feedback which serves to strengthen the community of care providers and improve patient outcomes. As such, PD appears to be a promising technique for data-driven actionable feedback.

Benchmarking

The concept of benchmarking was originally found in economics as a QI tool, where it was described as a continuous systematic process for comparative assessment of high-level performance, for purposes of organizational improvement (69). By identifying the best performance (i.e., the ‘benchmark’), the process of benchmarking can facilitate an understanding of the processes leading to success (top performers), and compare institutional-level practices with competitors, leading to modifications of behaviour/practices and subsequently improvements. This leads to a cycle of continuous QI of comparing to, and learning from, the benchmark (70) (Figure 3).

Figure 3 Benchmarking cycle. Adapted from (70) with permission.

More recently, the concept has since been applied to the surgical field. Here, benchmarking serves to identify top performers within a hospital, both locally, regionally, or even internationally, serving to facilitate surgical QI by way of comparison with a realistic optimal outcome (70). There are ten key steps in selecting a valid benchmark. These include: (I) Intervention: selecting the intervention desired for benchmarking (i.e., esophagectomy). (II) Patients: specify patient cohort for which beset possible outcome is expected (i.e., patients to who represent the lowest risk of complications). (III) Outcome(s): define which outcomes (benchmarks) to be measured and process of measuring them. (IV) Centres: involve centres which are eligible for benchmarking (in general, such centres should be high-volume, participate in prospective databases, and contribute to frequent research in the field of interest). (V) Number: include multiple centres from at least 3 continents, potentially reflecting a more realistic reflection of best outcome. (VI) Contact: leaders contact candidate centres for participation. (VII) Extract: extract the predefined patients with the lowest expected postoperative AEs of each centre. (VIII) Collect: collect centre-level data on patient characteristics and benchmark values for selected intervention. (IX) Calculate: calculate centre-level median (continuous variable benchmark value) or proportion (binary variable) of each benchmark value. (X) Benchmark: determine the benchmark value by computing the 75^th percentile for selected variable using each centre’s median. The 75^th percentile is selected as this has been identified as an achievable and realistic outcome marker by the majority of centres (70).

Using a Delphi consensus process, the potential surgical outcomes within the benchmarking process are identified and agreed upon (71). The final result of a benchmarking practice allows for the identification of potential actionable items aimed at decreasing the gap between hospitals reaching the 75^th percentile of benchmarking, and those which surpass this. An excellent example of benchmarking complications associated with esophagectomy was performed by Low et al. (72). Here, the authors used a standardized international dataset with specific definitions of AEs to benchmark complications and postoperative outcomes in patients undergoing esophagectomy for cancer. Twenty-four high-volume esophageal cancer centres from 14 countries were involved in the prospective data collection. The authors were able to define percentage of major complications, by Clavien-Dindo grade, as well as readmission rate and 30- and 90-day mortality. These results allow for a standardized method of establishing international benchmarks for reporting perioperative outcomes following esophagectomy. Knowing well the impact which AEs have on patients, their families, the cost to the healthcare system and society in general, benchmarking can serve to reduce postoperative AEs by identifying and comparing the optimal (and reachable) outcome of interest within each institution. This practice can lead to improvement in patient care by targeting areas which lend themselves to modification, enhancement and potentially standardization of patient perioperative pathways, which can be scaled up having maximal patient-centered impacts (73,74).

Conclusions

To improve, one must be able to measure AEs reliably, reproducibly and continuously, and then feedback outcomes to providers in a way that facilitates practice change. Feedback can be provided through multiple approaches but is optimally performed in a data-driven and actionable way. AE monitoring must be focused on capturing AE incidence and severity with a high degree of fidelity; often requiring surgical teams for AE adjudication, to generate high resolution data which can inform centre- and surgeon-specific QI via a combined best-evidence and best-practice approach. This centre-specific QI may then inform regional QI, which may in turn inform higher-level national and international QI programs. To unlock this pathway for multi-institutional collaboration, a common language of AEs and monitoring methods is required. Future advancement in AE monitoring and feedback for QI will inevitably require the commitment of hospitals and healthcare networks to address resource limitations and develop robust data collection and feedback infrastructure. Given the extensive burden of AEs on patient outcomes and cost of care, healthcare institutions are incentivized both from an economic perspective as well as a value-based care perspective to aid in developing data-driven QI programs which can create sustained and system-wide improvements to surgical care.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Donna E. Maziak and Patrick J. Villeneuve) for the series “Comprehensive Lung Cancer Care: A Continuum” published in Current Challenges in Thoracic Surgery. The article has undergone external peer review.

Peer Review File: Available at https://ccts.amegroups.org/article/view/10.21037/ccts-23-21/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://ccts.amegroups.org/article/view/10.21037/ccts-23-21/coif). The series “Comprehensive Lung Cancer Care: A Continuum” was commissioned by the editorial office without any funding or sponsorship. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Chen FF, Zhang D, Wang YL, et al. Video-assisted thoracoscopic surgery lobectomy versus open lobectomy in patients with clinical stage I non-small cell lung cancer: a meta-analysis. Eur J Surg Oncol 2013;39:957-63. [Crossref] [PubMed]
2. Phillips JD, Merkow RP, Sherman KL, et al. Factors affecting selection of operative approach and subsequent short-term outcomes after anatomic resection for lung cancer. J Am Coll Surg 2012;215:206-15. [Crossref] [PubMed]
3. Seely AJ, Ivanovic J, Threader J, et al. Systematic classification of morbidity and mortality after thoracic surgery. Ann Thorac Surg 2010;90:936-42; discussion 942. [Crossref] [PubMed]
4. Amrock LG, Deiner S. The implication of frailty on preoperative risk assessment. Curr Opin Anaesthesiol 2014;27:330-5. [Crossref] [PubMed]
5. Reames BN, Ghaferi AA, Birkmeyer JD, et al. Hospital volume and operative mortality in the modern era. Ann Surg 2014;260:244-51. [Crossref] [PubMed]
6. Ivanovic J, Seely AJ, Anstee C, et al. Measuring surgical quality: comparison of postoperative adverse events with the american college of surgeons NSQIP and the Thoracic Morbidity and Mortality classification system. J Am Coll Surg 2014;218:1024-31. [Crossref] [PubMed]
7. Khan NA, Quan H, Bugar JM, et al. Association of postoperative complications with hospital costs and length of stay in a tertiary care center. J Gen Intern Med 2006;21:177-80. [Crossref] [PubMed]
8. Khuri SF, Henderson WG, DePalma RG, et al. Determinants of long-term survival after major surgery and the adverse effect of postoperative complications. Ann Surg 2005;242:326-41; discussion 341-3. [Crossref] [PubMed]
9. Andalib A, Ramana-Kumar AV, Bartlett G, et al. Influence of postoperative infectious complications on long-term survival of lung cancer patients: a population-based cohort study. J Thorac Oncol 2013;8:554-61. [Crossref] [PubMed]
10. Irshad K, Feldman LS, Chu VF, et al. Causes of increased length of hospitalization on a general thoracic surgery service: a prospective observational study. Can J Surg 2002;45:264-8. [PubMed]
11. Freeman RK, Dilts JR, Ascioti AJ, et al. A comparison of length of stay, readmission rate, and facility reimbursement after lobectomy of the lung. Ann Thorac Surg 2013;96:1740-5; discussion 1745-6. [Crossref] [PubMed]
12. Nojiri T, Hamasaki T, Inoue M, et al. Long-Term Impact of Postoperative Complications on Cancer Recurrence Following Lung Cancer Surgery. Ann Surg Oncol 2017;24:1135-42. [Crossref] [PubMed]
13. Zhang Z, Mostofian F, Ivanovic J, et al. All grades of severity of postoperative adverse events are associated with prolonged length of stay after lung cancer resection. J Thorac Cardiovasc Surg 2018;155:798-807. [Crossref] [PubMed]
14. Harrison R, Walton M, Manias E, et al. The missing evidence: a systematic review of patients' experiences of adverse events in health care. Int J Qual Health Care 2015;27:424-42. [Crossref] [PubMed]
15. Grigor EJM, Ivanovic J, Anstee C, et al. Impact of Adverse Events and Length of Stay on Patient Experience After Lung Cancer Resection. Ann Thorac Surg 2017;104:382-8. [Crossref] [PubMed]
16. Liu JB, Berian JR, Chen S, et al. Postoperative Complications and Hospital Payment: Implications for Achieving Value. J Am Coll Surg 2017;224:779-786.e2. [Crossref] [PubMed]
17. Vonlanthen R, Slankamenac K, Breitenstein S, et al. The impact of complications on costs of major surgical procedures: a cost analysis of 1200 patients. Ann Surg 2011;254:907-13. [Crossref] [PubMed]
18. Meltzer D. Economic analysis in patient safety: a neglected necessity. BMJ Qual Saf 2012;21:443-5. [Crossref] [PubMed]
19. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370-6. [Crossref] [PubMed]
20. Baker GR, Norton PG, Flintoft V, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004;170:1678-86. [Crossref] [PubMed]
21. Aucoin SD, Hao M, Sohi R, et al. Accuracy and Feasibility of Clinically Applied Frailty Instruments before Surgery: A Systematic Review and Meta-analysis. Anesthesiology 2020;133:78-95. [Crossref] [PubMed]
22. McIsaac DI, Taljaard M, Bryson GL, et al. Frailty as a Predictor of Death or New Disability After Surgery: A Prospective Cohort Study. Ann Surg 2020;271:283-9. [Crossref] [PubMed]
23. Jones D, Kumar S, Anstee C, et al. Index hospital cost of adverse events following thoracic surgery: a systematic review of economic literature. BMJ Open 2023;13:e069382. [Crossref] [PubMed]
24. Ferlay J, Soerjomataram I, Dikshit R, et al. Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer 2015;136:E359-86. [Crossref] [PubMed]
25. Shamji FM, Gingrich M, Anstee C, et al. Standardized Postoperative Adverse Event Data Collection to Document, Inform, and Improve Patient Care. Thorac Surg Clin 2021;31:441-8. [Crossref] [PubMed]
26. Finley CJ, Bendzsak A, Tomlinson G, et al. The effect of regionalization on outcome in pulmonary lobectomy: a Canadian national study. J Thorac Cardiovasc Surg 2010;140:757-63. [Crossref] [PubMed]
27. Backman C, Vanderloo S, Forster AJ. Measuring and improving quality in university hospitals in Canada: The Collaborative for Excellence in Healthcare Quality. Health Policy 2016;120:982-6. [Crossref] [PubMed]
28. The global, regional, and national burden of oesophageal cancer and its attributable risk factors in 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet Gastroenterol Hepatol 2020;5:582-97. [Crossref] [PubMed]
29. Habbous S, Yermakhanova O, Forster K, et al. Variation in Diagnosis, Treatment, and Outcome of Esophageal Cancer in a Regionalized Care System in Ontario, Canada. JAMA Netw Open 2021;4:e2126090. [Crossref] [PubMed]
30. Fuchs HF, Harnsberger CR, Broderick RC, et al. Mortality after esophagectomy is heavily impacted by center volume: retrospective analysis of the Nationwide Inpatient Sample. Surg Endosc 2017;31:2491-7. [Crossref] [PubMed]
31. Clark JM, Boffa DJ, Meguid RA, et al. Regionalization of esophagectomy: where are we now? J Thorac Dis 2019;11:S1633-42. [Crossref] [PubMed]
32. Birkmeyer JD, Stukel TA, Siewers AE, et al. Surgeon volume and operative mortality in the United States. N Engl J Med 2003;349:2117-27. [Crossref] [PubMed]
33. Finley C, Schneider L, Shakeel S. Approaches to high-risk, resource intensive cancer surgical care in Canada. Canadian Partnership Against Cancer. 2019.
34. Miller JD, Jain MK, de Gara CJ, et al. Effect of surgical experience on results of esophagectomy for esophageal carcinoma. J Surg Oncol 1997;65:20-1. [Crossref] [PubMed]
35. Dindo D, Demartines N, Clavien PA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 2004;240:205-13. [Crossref] [PubMed]
36. Katayama H, Kurokawa Y, Nakamura K, et al. Extended Clavien-Dindo classification of surgical complications: Japan Clinical Oncology Group postoperative complications criteria. Surg Today 2016;46:668-85. [Crossref] [PubMed]
37. Yoon PD, Chalasani V, Woo HH. Use of Clavien-Dindo classification in reporting and grading complications after urological surgical procedures: analysis of 2010 to 2012. J Urol 2013;190:1271-4. [Crossref] [PubMed]
38. Monteiro E, Sklar MC, Eskander A, et al. Assessment of the Clavien-Dindo classification system for complications in head and neck surgery. Laryngoscope 2014;124:2726-31. [Crossref] [PubMed]
39. Bolliger M, Kroehnert JA, Molineus F, et al. Experiences with the standardized classification of surgical complications (Clavien-Dindo) in general surgery patients. Eur Surg 2018;50:256-61. [Crossref] [PubMed]
40. Ivanovic J, Al-Hussaini A, Al-Shehab D, et al. Evaluating the reliability and reproducibility of the Ottawa Thoracic Morbidity and Mortality classification system. Ann Thorac Surg 2011;91:387-93. [Crossref] [PubMed]
41. Salati M, Refai M, Pompili C, et al. Major morbidity after lung resection: a comparison between the European Society of Thoracic Surgeons Database system and the Thoracic Morbidity and Mortality system. J Thorac Dis 2013;5:217-22. [PubMed]
42. Yang J, Xia Y, Yang Y, et al. Risk factors for major adverse events of video-assisted thoracic surgery lobectomy for lung cancer. Int J Med Sci 2014;11:863-9. [Crossref] [PubMed]
43. Sandri A, Papagiannopoulos K, Milton R, et al. Major morbidity after video-assisted thoracic surgery lung resections: a comparison between the European Society of Thoracic Surgeons definition and the Thoracic Morbidity and Mortality system. J Thorac Dis 2015;7:1174-80. [PubMed]
44. Sigler G, Anstee C, Seely AJE. Harmonization of adverse events monitoring following thoracic surgery: Pursuit of a common language and methodology. JTCVS Open 2021;6:250-6. [Crossref] [PubMed]
45. Seely AJE, Fowler R. Importance of nationally coordinated data collection and quality improvement networks. Can J Respir Crit Care Sleep Med 2022;6:6-8. [Crossref]
46. Orlander JD, Barber TW, Fincke BG. The morbidity and mortality conference: the delicate nature of learning from error. Acad Med 2002;77:1001-6. [Crossref] [PubMed]
47. Churchill KP, Murphy J, Smith N. Quality Improvement Focused Morbidity and Mortality Rounds: An Integrative Review. Cureus 2020;12:e12146. [Crossref] [PubMed]
48. Kwok ES. Inadequacy of Current Morbidity and Mortality Review Activities: Evolution of a Time-Honored Tradition Needed. Jt Comm J Qual Patient Saf 2017;43:3-4. [Crossref] [PubMed]
49. Moskowitz E, Veloski JJ, Fields SK, et al. Development and evaluation of a 1-day interclerkship program for medical students on medical errors and patient safety. Am J Med Qual 2007;22:13-7. [Crossref] [PubMed]
50. Higginson J, Walters R, Fulop N. Mortality and morbidity meetings: an untapped resource for improving the governance of patient safety? BMJ Qual Saf 2012;21:576-85. [Crossref] [PubMed]
51. Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002;346:1128-37. [Crossref] [PubMed]
52. Marra AR, Guastelli LR, de Araújo CM, et al. Positive deviance: a new strategy for improving hand hygiene compliance. Infect Control Hosp Epidemiol 2010;31:12-20. [Crossref] [PubMed]
53. Wray JD. Editorial: Can we learn from successful mothers? J Trop Pediatr Environ Child Health 1972;18:279. [PubMed]
54. Wishik SM, Vynckt S. The use of nutritional 'positive deviants' to identify approaches for modification of dietary practices. Am J Public Health 1976;66:38-42. [Crossref] [PubMed]
55. Sternin M, Sternin J, Marsh D. Rapid, Sustained Childhood Malnutrition Alleviation through a Positive-Deviance Approach in Rural Vietnam: Preliminary Findings. In: Wollinka O, Keeley E, Burkhalter B, Bashir N, editors. Hearth Nutrition Model: Applications in Haiti, Vietnam, and Bangladesh. Basic Support for Institutionalizing Child Survival (BASICS) Project; 1996:59-76.
56. Marsh DR, Schroeder DG, Dearden KA, et al. The power of positive deviance. BMJ 2004;329:1177-9. [Crossref] [PubMed]
57. Choi H, Jung I, Lee Y. The power of positive deviance behaviours: From panic-gogy to effective pedagogy in online teaching. Educ Inf Technol 2023;28:12651-69. [Crossref] [PubMed]
58. Seeley KML, Foster BA, Zuckerman KE, et al. Positive deviance in the Oregon kindergarten assessment: Identifying schools and communities that are beating the odds. Early Child Res Q 2023;62:360-8. [Crossref]
59. Iyer P, Rolleston C, Huong VTT. Equitable, quality education for ethnic minority students? A case of “positive deviance” in vietnam. Comp Educ Rev 2021; [Crossref]
60. Whited T, Feng Y, Bruhn CM. Evaluation of the high school food safety curriculum using a positive deviance model. Food Control 2019;96:324-8. [Crossref]
61. Shija DS, Mwai OA, Migwi PK, et al. Identifying Positive Deviant Farms Using Pareto-Optimality Ranking Technique to Assess Productivity and Livelihood Benefits in Smallholder Dairy Farming under Contrasting Stressful Environments in Tanzania. World 2022;3:639-56. [Crossref]
62. Adelhart Toorop R, Ceccarelli V, Bijarniya D, et al. Using a positive deviance approach to inform farming systems redesign: A case study from Bihar, India. Agric Syst 2020;185:102942. [Crossref]
63. Ferracini FT, Marra AR, Schvartsman C, et al. Using Positive Deviance to reduce medication errors in a tertiary care hospital. BMC Pharmacol Toxicol 2016;17:36. [Crossref] [PubMed]
64. Sreeramoju P, Dura L, Fernandez ME, et al. Using a Positive Deviance Approach to Influence the Culture of Patient Safety Related to Infection Prevention. Open Forum Infect Dis 2018;5:ofy231. [Crossref] [PubMed]
65. Ivanovic J, Mostofian F, Anstee C, et al. Impact of Surgeon Self-evaluation and Positive Deviance on Postoperative Adverse Events After Non-cardiac Thoracic Surgery. J Healthc Qual 2018;40:e62-70. [Crossref] [PubMed]
66. Ivanovic J, Anstee C, Ramsay T, et al. Using Surgeon-Specific Outcome Reports and Positive Deviance for Continuous Quality Improvement. Ann Thorac Surg 2015;100:1188-94; discussion 1194-5. [Crossref] [PubMed]
67. Seely AJE, Ahmadzai Z, French DG, et al. Multicentre positive deviance to reduce adverse events and length of stay after pulmonary resection. BMJ Open Qual 2023;12:e001997. [Crossref] [PubMed]
68. Jones D, Gingrich M, Anstee C, et al. Multicentre positive deviance seminar to generate best practice recommendations to reduce anastomotic leak and length of stay for patients undergoing oesophagectomy. BMJ Open Qual 2023;12:e002458. [Crossref] [PubMed]
69. Ellis J. All inclusive benchmarking. J Nurs Manag 2006;14:377-83. [Crossref] [PubMed]
70. Staiger RD, Schwandt H, Puhan MA, et al. Improving surgical outcomes through benchmarking. Br J Surg 2019;106:59-64. [Crossref] [PubMed]
71. Gero D, Muller X, Staiger RD, et al. How to Establish Benchmarks for Surgical Outcomes?: A Checklist Based on an International Expert Delphi Consensus. Ann Surg 2022;275:115-20. [Crossref] [PubMed]
72. Low DE, Alderson D, Cecconello I, et al. International Consensus on Standardization of Data Collection for Complications Associated With Esophagectomy: Esophagectomy Complications Consensus Group (ECCG). Ann Surg 2015;262:286-94. [Crossref] [PubMed]
73. Barkun J, Clavien P, Pawlik TM. The Benefits of Benchmarking-A New Perspective on Measuring Quality in Surgery. JAMA Surg 2023;158:341-2. [Crossref] [PubMed]
74. Reintjes R, Thelen M, Reiche R, et al. Benchmarking national surveillance systems: a new tool for the comparison of communicable disease surveillance and control in Europe. Eur J Public Health 2007;17:375-80. [Crossref] [PubMed]