A multinational collaborative team led by the U.S. Pharmacopeial Convention is currently investigating the potential of near-infrared (NIR) spectroscopy for nontargeted detection of adulterants in skim and nonfat dry milk powder. The development of a compendial method is challenged by the range of authentic or nonadulterated milk powders available worldwide. This paper investigates the sources of variance in 41 authentic bovine skim and nonfat milk powders as detected by NIR diffuse reflectance spectroscopy and chemometrics. Exploratory analysis by principal component analysis and varimax factor rotation revealed significant variance in authentic samples and highlighted outliers from a single manufacturer. Spectral preprocessing and outlier removal methods reduced ambient and measurement sources of variance, most likely linked to changes in moisture together with sampling, preparation, and presentation factors. Results indicate that significant chemical variance exists in different skim and nonfat milk powders that will likely affect the performance of adulterant detection methods by NIR spectroscopy.