Human aging affects the ability to remember new experiences, in part, because of altered neural function during memory formation. One potential contributor to age-related memory decline is diminished neural selectivity -- i.e., a decline in the differential response of cortical regions to preferred vs. non-preferred stimuli during event perception -- yet the factors driving variability in neural selectivity with age remain unclear. We examined the impact of top-down attention and preclinical Alzheimer's disease (AD) pathology on neural selectivity during memory encoding in 156 cognitively unimpaired older participants who underwent fMRI while performing a word-face and word-scene associative memory task. Neural selectivity in face- and place-selective cortical regions was greater during events that were later remembered compared to forgotten. Critically, neural selectivity during learning positively scaled with memory-related variability in top-down attention, whereas selectivity negatively related to early AD pathology, evidenced by elevated plasma pTau181. Path analysis revealed that neural selectivity at encoding mediated the effects of age, top-down attention, and pTau181 on associative memory. Collectively, these data reveal multiple pathways that contribute to memory differences among older adults -- AD-independent reductions in top-down attention and AD-related pathology alter the precision of cortical representations of events during experience, with consequences for remembering.