Perceptual rating scales are widely used in voice quality assessment, yet apart from the GRBAS scale, their reliability has been poorly demonstrated. There are no studies that have compared the optimal reliability of experienced judges using different auditory rating scales in a controlled experimental environment. This study aimed to assess the reliability of three common scales (The Buffalo Voice Profile, The Vocal Profile Analysis Scheme (VPA) and GRBAS. Sixty-five varyingly dysphonic and five normal voices were recorded onto CD in random order. Thirty voices were recorded twice. Seven experienced and trained speech and language therapists rated all voices on the three scales. Only the overall grade was found to be reliable for the Buffalo Voice Profile. The reliability of the VPA scheme was found to be poor to moderate. The VPA may have a use as a multi-dimensional and in-depth evaluation of voice types, but its greater scope is at the expense of reliability. The GRBAS was reliable across all parameters except Strain. Our detailed reliability analysis comparing performance of three commonly used rating scales provides further evidence to support the GRBAS as a simple reliable measure for clinical use.