In this paper we describe a methodology for measuring the ground truth for boundary detection methods in medical images. An interface is used to allow medical experts to outline boundaries of interest on images. Following this, the accuracy of the experts in repeating the ground truth detection process is evaluated. Finally, an approach is described for comparing the ground truth as provided by experts with actual boundaries detected automatically. Experimental results are presented to validate the proposed algorithms.