While age-associated B cells (ABCs) are known to expand and persist following viral infection and during autoimmunity, their interactions are yet to be studied together in these contexts. Here, we directly compared CD11c+T-bet+ ABCs using models of Epstein-Barr virus (EBV), gammaherpesvirus 68 (γHV68), multiple sclerosis (MS), and experimental autoimmune encephalomyelitis (EAE), and found that each drives the ABC population to opposing phenotypes. EBV infection has long been implicated in MS, and we have previously shown that latent γHV68 infection exacerbates EAE. Here, we demonstrate that ABCs are required for γHV68-enhanced disease. We then show that the circulating ABC population is expanded and phenotypically altered in people with relapsing MS. In this study, we show that viral infection and autoimmunity differentially affect the phenotype of ABCs in humans and mice, and we identify ABCs as functional mediators of viral-enhanced autoimmunity.