Here, we describe the measurement characteristics of a 13-item dietary screener used in the Prostate Cancer Prevention Trial. We used data from 10,913 men who completed the 13-item dietary screener, a food frequency questionnaire (FFQ), and questionnaires on demographic and health-related characteristics and from 146 men who also completed multiple 24-h dietary recalls in a substudy. The analyses in this report focused on percentage energy from fat and saturated fat and used the mean estimates from the dietary recalls as the criterion measures. Absolute nutrient estimates from the screener were about one-third of the estimates from the recalls and the FFQ. Validity was defined as the Pearson correlation of the criterion measures of fat with the corresponding measures from the FFQ and the screener. The FFQ was a statistically significantly more precise measure of percentage energy from fat (r = 0.71) and saturated fat (r = 0.72) than was the screener (r = 0.50 and 0.53, respectively). There were also statistically significant differences in how well these instruments could detect variation in dietary fat across various participant characteristics, suggesting that the screener may not perform as well as the FFQ across demographic strata such as education (P < 0.001). The results from this study suggest that the use of short dietary screeners as the sole assessment instrument may result in a serious loss of information regarding important exposures (e.g., fat intake) and lost opportunities to enhance our knowledge regarding dietary factors and cancer risk.