Objective: Preoperative MRI of fistula in ano is becoming more common. This prospective study aimed to determine if a significant difference occurred in interpretation between one expert and one novice observer and to assess inter- and intraobserver agreement after both observers underwent a period of directed education. SUBJECTS AND METHODS. An outcome-derived reference standard was defined in 100 patients with suspected fistula in ano via a combination of preoperative MRI, surgical findings, and clinical outcome. The performances of a single expert and a single novice interpreter were compared with this reference standard both before and after a period of directed education, and inter- and intraobserver agreement was determined.
Results: Initially the expert correctly classified significantly more fistulas than the novice (85% vs 63%, p = 0.024), but after directed education there was no significant difference, with good agreement for both the classification of the primary track (kappa = 0.71) and identification of extensions (k = 0.61). Intraobserver agreement was very good for the expert (kappa = 0.92) and novice (kappa = 0.88) for classification of the primary track and good (kappa = 0.64 and 0.74, respectively) for identification of extensions.
Conclusion: The diagnostic accuracy for fistula in ano classification using MRI was significantly higher for one expert than for one novice, though this was rectified by a short period of directed education.