Introduction: Interater agreement is a main aspect in the planning and conducting a clinical trial. The objective of this study is to show the application of assessing levels of interexaminer's agreement when multiple ratings are made on a single subject as an efficient method of evaluating the interater's reliability in planning a multicenter clinical trial.
Method: 39 psychiatrist assessed videotape of a patient with depression. The use of any reliability statistics is failed in the analysis of multiple independent ratings of a single subject since we used the methodology proposed by Cichetti et al. This method is able to identify raters whose ratings differ significantly from the average ratings and is sensitive for different levels of agreement.
Results: Interarter agreement was between good and excellent. The raters show an adequate agreement in total and items scores.
Conclusion: The design proposed permits evaluate the interater agreement in an efficient way (only in one session).