The USA leads other industrialized countries in health care spending but lags behind in terms of health outcomes. There has been growing interest in comparative effectiveness research (CER) as a means to identify best practices to create a more efficient and effective health care system. Two key concepts of CER are that it should (i) compare two or more alternative tests, therapies or procedures and (ii) be conducted in persons, clinical settings and conditions that are representative of the real world. The goal of CER is to provide evidence for clinicians, patients, policy makers and others to make informed decisions that will ultimately improve the overall health of specific subgroups and of the population as a whole. In this narrative review, we first describe the strengths and limitations of various types of studies that constitute CER, including randomized clinical trials, observational studies and systematic reviews, providing examples from the nephrology literature. Because of the concerns regarding confounding in observational CER, we also provide an overview of methods to reduce confounding in these types of studies. Finally, we will discuss why CER pertaining to kidney disease care needs to be a top priority in order to move our field from a largely opinion-based specialty to an evidence-based specialty.