University of Wisconsin Population Health Institute has published County Health Rankings (The Rankings) since 2010. These rankings use population-based data to highlight variation in health and encourage health assessment for all US counties. However, the uncertainty of estimates remains a limitation. We sought to quantify the precision of The Rankings for selected measures. We developed hierarchical models for 5 health outcome measures and applied empirical Bayes methods to obtain county rank estimates for a composite health outcome measure. We compared results using models with and without demographic fixed effects to determine whether covariates improved rank precision. Counties whose rank had wide confidence intervals had smaller populations or ranked in the middle of all counties for health outcomes. Incorporating covariates in the models produced narrower intervals, but rank estimates remained imprecise for many counties. Local health officials, especially in smaller population and mid-performing communities, should consider these limitations when interpreting the results of The Rankings.