Antiretroviral therapies have become widely available, and as a result, individuals infected with the human immunodeficiency virus (HIV) are living longer, and becoming integrated into the geriatric population. Around half of the HIV+ population shows some degree of cognitive impairment, but it is unknown how their neural networks and brain connectivity compare to those of noninfected people. Here we combined magnetic resonance imaging-based cortical parcellations with high angular resolution diffusion tensor imaging tractography in 55 HIV-seropositive patients and 30 age-matched controls, to map white matter connections between cortical regions. We set out to determine selective virus-associated disruptions in the brain's structural network. All individuals in this study were aged 60-80, with full access to antiretroviral therapy. Frontal and motor connections were compromised in HIV+ individuals. HIV+ people who carried the apolipoprotein E4 allele (ApoE4) genotype-which puts them at even greater risk for neurodegeneration-showed additional network structure deficits in temporal and parietal connections. The ApoE4 genotype interacted with duration of illness. Carriers showed greater brain network inefficiencies the longer they were infected. Neural network deficiencies in HIV+ populations exceed those typical of normal aging, and are worse in those genetically predisposed to brain degeneration. This work isolates neuropathological alterations in HIV+ elders, even when treated with antiretroviral therapy. Network impairments may contribute to the neuropsychological abnormalities in elderly HIV patients, who will soon account for around half of all HIV+ adults.