Individuals differ in how they perceive, remember, and think. There is evidence for the existence of distinct subgroups that differ in cognitive performance within the older population. However, it is less clear how individual differences in cognition in old age are linked to differences in brain-based measures. We used latent-profile analysis on n-back working-memory (WM) performance to identify subgroups in a large sample of older adults (n = 181; age = 64-68 years). Our analysis identified one larger normal subgroup with higher performance (n = 113; 63%), and a second smaller subgroup (n = 55; 31%) with lower performance. The low-performing subgroup showed weaker load-dependent BOLD modulation and lower connectivity within the fronto-parietal network (FPN) as well as between FPN and striatum during n-back, along with lower FPN connectivity at rest. This group also exhibited lower FPN structural integrity, lower frontal dopamine D2 binding potential, inferior performance on offline WM tests, and a trend-level genetic predisposition for lower dopamine-system efficiency. By contrast, this group exhibited relatively intact episodic memory and associated brain measures (i.e., hippocampal volume, structural, and functional connectivity within the default-mode network). Collectively, these data provide converging evidence for the existence of a group of older adults with impaired WM functioning characterized by reduced cortico-striatal coupling and aberrant cortico-cortical integrity within FPN.