The prevalence and costs of pain in the United States and throughout the world are high. This demonstrates the importance and need for comparative effectiveness research in pain. Comparative effectiveness research has been defined by the Institute of Medicine as evidence that compares the benefits and harms of treatment alternatives. Decision makers and consumers alike need comparative effectiveness data to aid in the decision making process. However, comparative effectiveness research presents many challenges and, in pain, has primarily been conducted in randomized controlled trials. These trials have some limitations that may prevent the application to the general population. Real world studies complement the information from randomized trials, but contain their own challenges and limitations. The International Society of Pharmacoeconomics and Outcomes Research has provided guidelines for the use of real world studies in comparative effectiveness research. The purpose of this article is to describe and address some the key methodological challenges of comparative effectiveness research, especially in pain.