Background: The sociomedical evaluation by the German Pension Insurance serves the purpose of determining entitlement to disability pensions. A quality assurance concept for the sociomedical evaluation was developed, which is based on a peer Review process. Peer review is an established process of external quality assurance in health care. The review is based on a hierarchically constructed manual that was evaluated in this pilot project.
Methods: The database consists of 260 medical reports for disability pension of 12 pension insurance agencies. 771 reviews from 19 peers were included in the evaluation of the inter-rater reliability. Kendall's coefficient of concordance W for more than 2 raters is used as primary measure of inter-rater reliability.
Results: Reliability appeared to be heterogeneous. Kendalls W varies for the particular criteria from 0.09 to 0.88 and reached for primary criterion reproducibility a value of 0.37.
Conclusion: The reliability of the manual seemed acceptable in the context of existing research data and is in line with existing peer review research outcomes. Nevertheless, the concordance is limited and requires optimisation. Starting points for improvement can be seen in a systematic training and regular user meetings of the peers involved.
© Georg Thieme Verlag KG Stuttgart · New York.