Background: Stigmata of hemorrhage predict rebleeding and outcome of patients with bleeding peptic ulcers. There are variabilities in reported incidences of stigmata and their respective rebleeding risks. We sought to study the interobserver agreement among experts.
Methods: Between June 1994 and July 1994, 100 consecutive patients with bleeding peptic ulcers underwent videoendoscopy within 24 hours of their admissions. An edited videotape of these ulcers was compiled and sent to an international panel of 14 experts. They independently rated these ulcers exclusively into one of the six categories: spurting, oozing, nonbleeding visible vessel, adherent clot, flat pigmented spot, or clean based. Agreement between any two experts was expressed by a kappa estimate (kappa). Agreements over individual stigmata and a composite kappa estimate (kappa(w)) signifying overall agreement were also computed.
Results: Out of the possible 91 pairwise kappa estimates among 14 experts, 35 (38.5%) were less than or equal to 0.40, indicating poor agreement. None of the kappa estimates was greater than 0.75. Composite kappa estimates for individual stigmata were as follows: spurting kappa = 0.664, oozing kappa = 0.420, nonbleeding visible vessel kappa = 0.342, adherent clot kappa = 0.426, flat pigmented spot kappa = 0.393, and clean-based ulcer kappa = 0.371. The weighted kappa estimate was 0.426.
Conclusion: Agreement between experts was poor in more than a third of occasions. Although the overall interobserver agreement was fair (0.4 < kappa < 0.75), agreements for nonbleeding visible vessels, flat pigmented spots, and clean-based ulcers were poor.