Multirater agreement of the causes of anterior cruciate ligament reconstruction failure: a radiographic and video analysis of the MARS cohort

Background: Anterior cruciate ligament (ACL) reconstruction failure occurs in up to 10% of cases. Technical errors are considered the most common cause of graft failure despite the absence of validated studies. Limited data are available regarding the agreement among orthopaedic surgeons regarding the causes of primary ACL reconstruction failure and accuracy of graft tunnel placement.

Hypothesis: Experienced knee surgeons have a high level of interobserver reliability in the agreement about the causes of primary ACL reconstruction failure, anatomic graft characteristics, and tunnel placement.

Study design: Cohort study (diagnosis); Level of evidence, 3.

Methods: Twenty cases of revision ACL reconstruction were randomly selected from the Multicenter ACL Revision Study (MARS) database. Each case included the patient's history, standardized radiographs, and a concise 30-second arthroscopic video taken at the time of revision demonstrating the graft remnant and location of the tunnel apertures. All 20 cases were reviewed by 10 MARS surgeons not involved with the primary surgery. Each surgeon completed a 2-part questionnaire dealing with each surgeon's training and practice, as well as the placement of the femoral and tibial tunnels, condition of the primary graft, and the surgeon's opinion as to the causes of graft failure. Interrater agreement was determined for each question with the kappa coefficient and the prevalence-adjusted, bias-adjusted kappa (PABAK).

Results: The 10 reviewers have been in practice an average of 14 years and have performed at least 25 ACL reconstructions per year, and 9 were fellowship trained in sports medicine. There was wide variability in agreement among knee experts as to the specific causes of ACL graft failure. When participants were specifically asked about technical error as the cause for failure, interobserver agreement was only slight (PABAK = 0.26). There was fair overall agreement on ideal femoral tunnel placement (PABAK = 0.55) but only slight agreement on whether a femoral tunnel was too anterior (PABAK = 0.24) and fair agreement on whether it was too vertical (PABAK = 0.46). There was poor overall agreement for ideal tibial tunnel placement (PABAK = 0.17).

Conclusion: This study suggests that more objective criteria are needed to accurately determine the causes of primary ACL graft failure as well as the ideal femoral and tibial tunnel placement in patients undergoing revision ACL reconstruction.