The immune checkpoint pathway of human programmed cell death 1 (hPD-1) and human programmed cell death ligand 1 (hPD-L1) is a promising target for cancer treatment. The blockade of the interplay between hPD-1 and hPD-L1 has recently shown good therapeutic efficacy. Although crystallographic studies have provided static conformational snapshots of the interface between hPD-1 and hPD-L1, the hot spot residues on both proteins that play key roles in the association process still remain elusive. To this end, we performed a series of alchemical free-energy simulations to analyze the energetic contributions of the interfacial residues on both hPD-1 and hPD-L1 and investigated the distributional patterns of the residues that significantly contribute to the binding. The results suggest that the hot spots on hPD-1 comprise Tyr68, Gln75, Ile126, Leu128, Ile134, and Glu136, and the hot spots on hPD-L1 comprise LAsp26 (the L symbol refers to hPD-L1), LIle54, LTyr56, LMet115, LAsp122, LTyr123, and LLys124. Moreover, we found that the distribution of these hot spot residues is highly uneven with respect to either the energetic contribution or the side-chain polarity, with energetically important residues clustered within densely packed hydrophobic regions. The mechanism ruling the interaction of the two binding partners is also discussed in detail from the perspective of the O-ring theory. Our work provides clues for the future development of anticancer inhibitors targeting the hPD-1/hPD-L1 immune checkpoint pathway.