Reproducibility of Brain MRI Segmentation Algorithms: Empirical Comparison of Local MAP PSTAPLE, FreeSurfer, and FSL-FIRST

Background and purpose: Segmentation of human brain structures is crucial for the volumetric quantification of brain disease. Advances in algorithmic approaches have led to automated techniques that save time compared to interactive methods. Recently, the utility and accuracy of template library fusion algorithms, such as Local MAP PSTAPLE (PSTAPLE), have been demonstrated but there is little guidance regarding its reproducibility compared to single template-based algorithms such as FreeSurfer and FSL-FIRST.

Methods: Eight repeated magnetic resonance imagings of 20 subjects were segmented using FreeSurfer, FSL-FIRST, and PSTAPLE. We reported the reproducibility of segmentation-derived volume measurements for brain structures and calculated sample size estimates for detecting hypothetical rates of tissue atrophy given the observed variances.

Results: PSTAPLE had the most reproducible volume measurements for hippocampus, putamen, thalamus, caudate, pallidum, amygdala, Accumbens area, and cortical regions. FreeSurfer was most reproducible for brainstem. PSTAPLE was the most accurate algorithm in terms of several metrics include Dice's coefficient. The sample size estimates showed that a study utilizing PSTAPLE would require tens to hundreds less subjects than the other algorithms for detecting atrophy rates typically observed in brain disease.

Conclusions: PSTAPLE is a useful tool for automatic human brain segmentation due to its precision and accuracy, which enable the detection of the size of the effect typically reported for neurological disorders with a substantially reduced sample size, in comparison to the other tools we assessed. This enables randomized controlled trials to be executed with reduced cost and duration, in turn, facilitating the assessment of new therapeutic interventions.