The hallmark of HIV/SIV infections is the progressive depletion of CD4+ T cells that ultimately renders the host incapable of defending against AIDS defining opportunistic infections and malignancies. Although many potential mechanisms have been proposed to explain CD4+ T cell loss, we review here the growing evidence that fibrotic 'scarring' and consequent damage to the lymphatic tissue niche contributes to CD4+ T cell decline and limits CD4+ T cell re-population with retroviral therapy.