Despite many years of potent antiretroviral therapy, latently infected cells and low levels of plasma virus have been found to persist in HIV-infected patients. The factors influencing this persistence and their relative contributions have not been fully elucidated and remain controversial. Here, we address these issues by developing and employing a simple, but mechanistic viral dynamics model. The model has two novel features. First, it assumes that latently infected T cells can undergo bystander proliferation without transitioning into active viral production. Second, it assumes that the rate of latent cell activation decreases with time on antiretroviral therapy due to the activation and subsequent loss of latently infected cells specific for common antigens, leaving behind cells that are successively less frequently activated. Using the model, we examined the quantitative contributions of T cell bystander proliferation, latent cell activation, and ongoing viral replication to the stability of the latent reservoir and persisting low-level viremia. Not surprisingly, proliferation of latently infected cells helped maintain the latent reservoir in spite of loss of latent infected cells through activation and death, and affected viral dynamics to an extent that depended on the magnitude of latent cell activation. In the limit of zero latent cell activation, the latent cell pool and viral load became uncoupled. However, as the activation rate increased, the plasma viral load could be maintained without depleting the latent reservoir, even in the absence of viral replication. The influence of ongoing viral replication on the latent reservoir remained insignificant for drug efficacies above the "critical efficacy" irrespective of the activation rate. However, for lower drug efficacies viral replication enabled the stable maintenance of both the latent reservoir and the virus. Our model and analysis methods provide a quantitative and qualitative framework for probing how different viral and host factors contribute to the dynamics of the latent reservoir and the virus, offering new insights into the principal determinants of their persistence.