Variability of interobserver interpretation of selected helminth ova in the development of a training image set

Background: Diagnosis of soil-transmitted helminthiasis and schistosomiasis for surveillance relies on microscopic detection of ova in Kato-Katz (KK) prepared slides. Artificial intelligence (AI)-based platforms for parasitic eggs may be developed using a robust image set with defined labels by reference microscopists. This study aimed to determine interobserver variability among reference microscopists in identifying parasite ova.

Methods: Images of parasite ova taken from KK prepared slides were labelled according to species by two reference microscopists (M1 and M2). A third reference microscopist (M3) labelled images when the first two did not agree. Frequency, percent agreement, κ statistics and variability score (VS) were generated for analysis.

Results: M1 and M2 agreed on 89.24% of the labelled images (κ=0.86, p<0.001). M3 had agreement with M1 and M2 (κ=0.30, p<0.001 and κ=0.28, p<0.001), resolving 89.29% of disagreement between them. The labelling of Schistosoma japonicum had the highest VS (κ=0.487, p=0.101) among the targeted ova. Reference microscopists were able to reliably reach consensus in 99.0% of the dataset.

Conclusions: Training AI using this image set may provide more objective and reliable readings compared with that of reference microscopists.