In a sample of first-admission psychotic patients, best-estimate diagnoses made by psychiatrists at entry to the study (N = 310) and 6 months later (N = 228) were compared with Structured Clinical Interview for DSM-III-R (SCID) algorithm diagnoses. Sensitivity, specificity, and agreement (kappa) at entry and at 6-month follow-up evaluation were satisfactory for schizophrenia (sensitivity, .89 and .98; specificity, .96 both times; kappa, .86 and .92) and bipolar disorder with psychosis (sensitivity, 1.00 and .94; specificity, .96 both times; kappa, .89 and .88), moderate for major depression with psychosis (sensitivity, .90 and .81; specificity, .94 and .95; kappa, .75 and .72), but mixed for the organic psychoses (sensitivity, .50 and .23; specificity, 1.00 both times; kappa, .66 and .36). Reasons for disagreement included the role of drugs and other organic factors in the etiology of the disorder, and clinical judgment versus the rules of the structured interview. We conclude that the SCID, when administered by closely supervised experienced nonpsychiatrist clinicians and incorporating information from other sources, can produce a reliable diagnosis of schizophrenia and bipolar disorder. However, the best-estimate procedure seems mandatory in studies investigating a broad range of psychoses, where the use of drugs is not an exclusion criterion.