The present study used functional and structural MRI to investigate differences in neuronal substrates underlying shifts of attention in young and old subjects, studied with dichotic listening. Two different consonant-vowel syllables were presented and the subjects were instructed to attend to and report from either the left or right ear stimulus. Typically, a right-ear advantage is observed when attending to the right-ear stimulus, and a left-ear advantage when attending to the left-ear stimulus. The behavioral results showed that the old group had difficulties with attentional modulation of the right-ear advantage in the attend left condition. This is interpreted as a failure of an important aspect of attentional control; the top-down biasing of attention for selection of task-relevant stimulus. The fMRI results showed that an area in the left middle frontal gyrus was more activated in the young group compared to the old group in the attend left condition. The structural MRI data showed reduced gray matter density of the same area in the old group. Based on these converging findings, we suggest that the left middle frontal gyrus plays an important role in top-down biasing of selecting task-relevant stimuli, and to inhibit processing of task-irrelevant stimuli. To our knowledge, this is one of the first studies addressing the question on how age-related changes in attentional processing is reflected in both functional and structural differences in the brain.