HIV is adept at avoiding naturally generated T cell responses; therefore, there is a need to develop HIV-specific T cells with greater potency for use in HIV cure strategies. Starting with a CD4-based chimeric antigen receptor (CAR) that was previously used without toxicity in clinical trials, we optimized the vector backbone, promoter, HIV targeting moiety, and transmembrane and signaling domains to determine which components augmented the ability of T cells to control HIV replication. This re-engineered CAR was at least 50-fold more potent in vitro at controlling HIV replication than the original CD4 CAR, or a TCR-based approach, and substantially better than broadly neutralizing antibody-based CARs. A humanized mouse model of HIV infection demonstrated that T cells expressing optimized CARs were superior at expanding in response to antigen, protecting CD4 T cells from infection, and reducing viral loads compared to T cells expressing the original, clinical trial CAR. Moreover, in a humanized mouse model of HIV treatment, CD4 CAR T cells containing the 4-1BB costimulatory domain controlled HIV spread after ART removal better than analogous CAR T cells containing the CD28 costimulatory domain. Together, these data indicate that potent HIV-specific T cells can be generated using improved CAR design and that CAR T cells could be important components of an HIV cure strategy.