Agricultural nonpoint source pollution remains a persistent environmental problem, despite the large amount of money that has been spent on its abatement. At local scales, agricultural best management practices (BMPs) have been shown to be effective at reducing nutrient and sediment inputs to surface waters. However, these effects have rarely been found to act in concert to produce measurable, broad-scale improvements in water quality. We investigated potential causes for this failure through an effort to develop recommendations for the use of riparian buffers in addressing nonpoint source pollution in Wisconsin. We used frequency distributions of phosphorus pollution at two spatial scales (watershed and field), along with typical stream phosphorus (P) concentration variability, to simulate benefit/cost curves for four approaches to geographically allocating conservation effort. The approaches differ in two ways: (1) whether effort is aggregated within certain watersheds or distributed without regard to watershed boundaries (dispersed), and (2) whether effort is targeted toward the most highly P-polluting fields or is distributed randomly with regard to field-scale P pollution levels. In realistic implementation scenarios, the aggregated and targeted approach most efficiently improves water quality. For example, with effort on only 10% of a model landscape, 26% of the total P load is retained and 25% of watersheds significantly improve. Our results indicate that agricultural conservation can be more efficient if it accounts for the uneven spatial distribution of potential pollution sources and the cumulative aspects of environmental benefits.