Morbidity and mortality after colorectal procedures: comparison of data from the American College of Surgeons case log system and the ACS NSQIP

Background: Improving the quality of surgical care depends upon collection of robust data. The American College of Surgeons Case Log System enables surgeons to self-report patient risk factors and outcomes. In contrast, the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) uses trained data abstractors to record similar data and uses a strict data collection methodology. The objective of this study was to assess bias in data entry for colorectal cases by comparing data in these 2 registries.

Study design: One year of NSQIP (July 1, 2008 to June 30, 2009) and 7 years of Case Log (2003 to 2010) data were examined. Colorectal cases were identified by current procedural terminology code. The frequencies of comparably defined variables were compared, and mortality models were developed using logistic regression. Observed and expected mortality rates were compared.

Results: Rates of most risk factor and outcome variables were significantly higher in NSQIP than those in Case Log. NSQIP had a higher unadjusted mortality rate (4.46% versus 3.69%, p < 0.001); however, the adjusted odds of mortality was significantly higher in Case Log (odds ratio 1.32, p < 0.05). The Case Log model overpredicted mortality in NSQIP by 22%, whereas the NSQIP model underpredicted mortality in Case Log by 12%.

Conclusions: Significant differences exist between risk factor and outcome data in NSQIP and Case Log for colorectal procedures. These differences demonstrate the need for standardized data collection methods, as is required by NSQIP, including use of standard definitions, adherence to a follow-up period for outcomes, and use of audits. These measures would improve the validity of using a self-reported database to evaluate and benchmark performance.