We compared the Alcohol-Related Problems Survey (ARPS), a new screening measure targeted at harmful and hazardous drinking among older individuals, to a criterion standard (CS) among 22 persons aged 65 and older who reported consuming at least one drink of alcohol in the previous 12 months. The CS was conducted by a study physician and research assistant. It assessed risks from alcohol use, and consisted of a structured review of each subject's medical record, clinical interview, physical examination, and an interview with a collateral informant. Analyses included descriptive statistics for demographic and health characteristics, inter-rater reliability, agreement between the ARPS and the CS, reasons for disagreement, and sensitivity and specificity of the ARPS as compared to the CS. Using Landis and Koch criteria, inter-rater reliability between two physicians for 11 subjects was substantial (weighted kappa 0.79), but agreement between the ARPS and the CS was only fair (weighted kappa 0.28). Reasons for disagreement included problems with ARPS' questions and classification rules, and problems with study physicians' assessments of drinking risk. Based on these reasons for disagreement, we made revisions in the ARPS and its classification rules. Agreement between the revised ARPS and the CS improved substantially (weighted kappa 0.62). Sensitivity and specificity of the original ARPS were 80% and 50%, respectively, and both improved to 82% after revisions. The revised ARPS is sensitive and specific for identifying harmful and hazardous drinking in older persons as determined by clinicians.