Aging is associated with progressive gray matter loss in the brain. This spatially specific, morphological change over the life span in humans is also found in chimpanzees, and the comparison between these great ape species provides a unique evolutionary perspective on human brain aging. Here, we present a data-driven, comparative framework to explore the relationship between gray matter atrophy with age and recent cerebral expansion in the phylogeny of chimpanzees and humans. In humans, we show a positive relationship between cerebral aging and cortical expansion, whereas no such relationship was found in chimpanzees. This human-specific association between strong aging effects and large relative cortical expansion is particularly present in higher-order cognitive regions of the ventral prefrontal cortex and supports the "last-in-first-out" hypothesis for brain maturation in recent evolutionary development of human faculties.