Brain derived neurotrophic factor (BDNF) seems to be involved in regulation of synaptic plasticity and neurogenesis. BDNF plasma and serum levels have been associated with depression, Alzheimer's disease, and other psychiatric and neurodegenerative disorders. In a community sample, drawn from the Baltimore Longitudinal Study of Aging (BLSA), we examined whether BDNF plasma concentration was associated with rates of age-related change in cognitive performance (n = 429) and regional brain volume (n = 59). Plasma BDNF levels, which were significantly higher in females (p<0.05), were not associated with either concurrent cognitive performance or rates of age-related change in performance across cognitive domains (p's>0.05). Sex differences in the relationship between BDNF and the trajectories of regional brain volume changes were observed for the whole brain and frontal white matter volumes (p<0.05), whereby lower plasma BDNF was associated with steeper volume decline in females but not males. Together, our findings contribute to furthering the understanding of the relationships between plasma BDNF, structural brain integrity and cognition. Potential mechanisms mediating these relationships merit further investigation.