The Treatment of Brain Arteriovenous Malformation Study (TOBAS): A preliminary inter- and intra-rater agreement study on patient management

Object: The best management of brain arteriovenous malformation (bAVM) patients remains unknown. Randomized allocation may be more readily accepted when there is uncertainty and disagreement regarding the management of potential participants. In planning for a trial, we aimed to assess variability and agreement among physicians managing bAVM patients.

Methods: A portfolio composed of 35 patients was sent to 47 clinicians of various specialties managing bAVM patients. For each patient, physicians were asked their best management decision (surgery/embolization/radiosurgery/conservative), their confidence level, and whether they would include the patient in a randomized trial comparing conservative and curative management. Seven physicians, who had access to all images of each patient, independently responded twice, to assess inter and intra-rater agreement using kappa statistics.

Results: The inter-rater agreement (30 raters, including 16 neuroradiologists) for best management decision was only "fair" (κ [95%CI]=0.210[0.157; 0.295]). Agreement remained below 'substantial' (κ<.6) between physicians of the same specialty, and when no distinctions were made between various treatments (when responses were dichotomized as conservative versus curative). With access to all images the inter-rater agreement remained fair. The intra-rater agreement reached "substantial" only for the dichotomized decisions. Responding clinicians were willing to include 54.4% of patients (mainly unruptured bAVMs) in a randomized trial.

Conclusion: There is a lack of agreement among clinicians involved in the management of bAVM patients. In this study a substantial proportion of clinicians were willing to offer randomized allocation of management options to a substantial number of patients.