The objective of this study was to examine the consistency of conclusions of Cochrane systematic reviews when different criteria are used to determine levels of evidence. We reanalyzed the data in six Cochrane reviews of conservative treatment of low back pain by applying three additional sets of "levels of evidence" criteria. Overall agreement between the conclusions attained with the different levels of evidence criteria was only "fair" (multirater kappa coefficient 0.33; 95% CI 0.28 to 0.38). For example, the four sets of levels of evidence criteria produced four conclusions on the efficacy of back school: "strong evidence that back schools are effective," "weak evidence," "limited evidence," and "no evidence." Pairwise agreement between the four pooling systems ranged from slight to substantial (kappas ranging from 0.10 to 0.80). Different rules for determining levels of evidence in systematic reviews produce markedly different conclusions on treatment efficacy. Crown