Lesions in the hippocampus (HC), the entorhinal cortex (EC), and the prefrontal cortex (PFC) are associated with impairment of episodic memory; reduced HC volume is linked to memory declines in dementia; and decline in EC volume predicts progression from mild cognitive impairment to dementia. However, in healthy adults, the relationship between memory and regional volumes is unclear, and no data are available on the relationship of longitudinal regional shrinkage to memory performance in a cognitively intact population. The objective of this study was to examine whether shrinkage of the EC, HC, and PFC over a 5 year period can predict declarative memory performance in healthy adults. The volumes of three brain regions were measured on magnetic resonance images that were acquired twice, 5 years apart. Multiple measures of episodic memory were administered at follow-up. Results indicated that the volume of HC and PFC (but not EC) correlated with age at baseline and follow-up. However, after age differences in memory were taken into account, none of the regional volumes was associated with memory performance at follow-up. In contrast, greater annual rate of shrinkage in EC (but not HC or PFC) predicted poorer memory performance. Thus, in a healthy and educated population, even mild age-related shrinkage of the EC may be a sensitive predictor of memory decline.