We investigated the relationship between vascular disease and risk factors versus cognitive decline cross-sectionally and longitudinally in normal older control, mild cognitive impairment, and mild Alzheimer disease (AD) dementia subjects. A total of 812 participants (229 normal older control, 395 mild cognitive impairment, 188 AD) underwent cognitive testing, brain magnetic resonance imaging, and clinical evaluations at baseline and over a period of 3 years. General linear, longitudinal mixed-effects, and Cox proportional hazards models were used. Greater homocysteine level and white matter hyperintensity volume were associated with processing speed impairment (homocysteine: P=0.02; white matter hyperintensity: P<0.0001); greater Vascular Index score was associated with memory impairment (P=0.007); and greater number of apolipoprotein E ε4 (APOE4) alleles was associated with global cognitive impairment (P=0.007) at baseline. Apolipoprotein E ε4 was associated with greater rate of increase in global cognitive impairment (P=0.002) and processing speed impairment (P=0.001) over time, whereas higher total cholesterol was associated with greater rate of increase in global cognitive impairment (P=0.02) and memory impairment (P=0.06) over time. These results suggest a significant association of increased vascular disease and risk factors with cognitive impairment at baseline and over time in the AD spectrum in a sample that was selected to have low vascular burden at baseline.