Peptide amphiphiles (PAs), functionalized with alkyl chains, are capable of self-assembling into various nanostructures. Recently, PAs have been considered as ideal drug carriers due to their good biocompatibility, specific biological functions, and hypotoxicity to normal cells and tissues. Meanwhile, the nanocarriers formed by PAs are able to achieve controlled drug release and enhanced cell uptake in response to the stimulus of the physiological environment or specific biological factors in the location of the lesion. However, the underlying detailed drug delivery mechanism, especially from the aspect of primary and secondary structures of PAs, has not been systematically summarized or discussed. Focusing on the relationship between the primary and secondary structures of PAs and stimuli-responsive drug delivery applications, this review highlights the recent advances, challenges, and opportunities of PA-based functional drug nanocarriers, and their potential pharmaceutical applications are discussed.