Background: Investigators aim to publish their research papers in top journals to disseminate their findings to the widest possible audience. Systematic reviews of the literature occupy the highest position in currently proposed hierarchies of evidence. We hypothesized that the number of citations (a measure of scholarly interest) for systematic reviews (or meta-analyses) published in leading orthopaedic journals would be greater than the number of citations for narrative reviews published in the same journals.
Methods: We identified fifteen journals that had high Science Citation Index impact factors for the orthopaedic subspecialty and were believed to have a higher yield of studies and reviews of scientific merit and clinical relevance. For the year 2000, six research associates applied methodological criteria to each article in each issue of the fifteen journals to determine whether the article was scientifically sound (rigorous versus nonrigorous). Of the 3916 articles identified, 2331 were original or review articles. We queried the ISI (Institute for Scientific Information) Web of Science database to ascertain, as of March 2003, the number of subsequent citations to each one of the reviews after its original publication in all journals that published both narrative and systematic reviews.
Results: Of the 2331 articles published across the fifteen journals in the year 2000, 110 were review articles. Only seventeen (15%) of the 110 reviews met our criteria for systematic reviews with rigor. Rigorous systematic reviews received more than twice the mean number of citations compared with other systematic or narrative reviews (13.8 compared with 6.0, p = 0.008). The rigor of a review was a significant predictor of the number of citations in other orthopaedic journals (p = 0.01). In addition, rigor was significantly associated with the number of citations in nonorthopaedic journals (p = 0.03).
Conclusions: Our findings suggest that journal editors and authors can improve the relevance and scholarly interest in their reviews (as shown by the number of citations) by meeting standard guidelines for methodological rigor.