Potyviruses are one of the most destructive viral pathogens of Solanaceae plants. In Capsicum annuum landrace CM334, a broad-spectrum gene, Pvr4 is known to be involved in resistance against multiple potyviruses, including Pepper mottle virus (PepMoV), Pepper severe mosaic virus (PepSMV), and Potato virus Y (PVY). However, a potyvirus avirulence factor against Pvr4 has not been identified. To identify the avirulence factor corresponding to Pvr4 in potyviruses, we performed Agrobacterium-mediated transient expressions of potyvirus protein coding regions in potyvirus-resistant (Pvr4) and -susceptible (pvr4) pepper plants. Hypersensitive response (HR) was observed only when a RNA-dependent RNA polymerase (NIb) of PepMoV, PepSMV, or PVY was expressed in Pvr4-bearing pepper leaves in a genotype-specific manner. In contrast, HR was not observed when the NIb of Tobacco etch virus (TEV), a virulent potyvirus, was expressed in Pvr4-bearing pepper leaves. Our results clearly demonstrate that NIbs of PepMoV, PepSMV, and PVY serve as avirulence factors for Pvr4 in pepper plants.