Diagnosis of progressive disease or (partial) response during tumor treatment is based on manual size estimates of enhancing tumor area: an expert measures two perpendicular diameters of the enhancing tumor region in a single MRI slice with the largest enhancing area. This paper analyzes the reliability of the area measure with respect to head placement in the MRI scanner and compares it with 3D volume measures in a dataset of eight subjects (5-7 follow-up scans each) with high-grade glioma. We show that the manual area measure is highly sensitive to head position changes, with a root mean squared error of 22%, compared to volume estimates with less than 5% error. In our simulated study using the 2D manual measurements, the majority of subjects would have been incorrectly diagnosed with progressive disease without any true anatomical changes. These results highlight the urgent need for revised and more reliable response assessment criteria, for example, based on increased slice resolution, 3D volume analysis and percent change computation with respect to an average of patient specific longitudinal measurements instead of a single measurement to define progression or response.