Multiple studies have broadened the roles of natural killer (NK) cells functioning as purely innate lymphocytes by demonstrating that they are capable of putative antigen-specific immunological memory against multiple infectious agents including HIV-1 and influenza. However, the mechanisms underlying antigen specificity remain unknown. Here, we demonstrate that antigen-specific human NK cell memory develops upon exposure to both HIV and influenza, unified by a conserved and epitope-specific targetable mechanism largely dependent on the activating CD94/NKG2C receptor and its ligand HLA-E. We validated the permanent acquisition of antigen specificity by individual memory NK cells by single-cell cloning. We identified elevated expression of KLRG1, α4β7, and NKG2C as biomarkers of antigen-specific NK cell memory through complex immunophenotyping. Last, we uncovered individual HLA-E-restricted peptides that may constitute the dominant NK cell response in HIV-1- and influenza-infected persons in vivo. Our findings clarify the mechanisms contributing to antigen-specific memory NK cell responses and suggest that they could be potentially targeted therapeutically for vaccines or other therapeutic interventions.