There is a need for more reliable and validated methods to assess the eye changes in Graves' disease. Such measurements are now available for the assessment of proptosis, eye muscle dysfunction, and optic nerve involvement. However, no validated objective measurement exists for NO SPECS class II signs. The present study compares the use of colour slides with clinical grading for assessing soft-tissue involvement. Forty-three patients were treated with retrobulbar irradiation. Pre-treatment, and 6 months post-treatment the severity of class II signs was graded from 0 to grade c in two ways; 1) Clinically, by two independent, experienced observers, who recorded their scores on the same day; 2) From colour slides, taken at the same visits, which were graded afterwards in one session independently by the same observers. Inter-observer agreement about clinical grading was low (Kappa 0.32), and was not improved by using the slides (Kappa 0.35). However, by using the clinical scores, the observers disagreed on treatment outcome in 21/43 patients (49%), whereas using the slides disagreement occurred in only 6/43 (14%, p < 0.01). It is concluded that the grading of soft-tissue involvement is highly subjective. However, the use of colour slides does provide a more reliable way to assess a treatment effect and should be used in clinical trials.