Inhibitor of apoptosis (IAP) proteins regulate programmed cell death by inhibiting members of the caspase family of proteases. The X-chromosome-linked IAP (XIAP) contains three baculovirus IAP repeat (BIR) domains, which bind directly to the N-termini of target proteins including those of caspases-3, -7, and -9. In the present study, we defined the consensus sequences of the motifs that interact with the three BIR domains in an unbiased manner. A combinatorial peptide library containing four random residues at the N-terminus was constructed and screened using BIR domains as probes. We found that the BIR3 domain binds a highly specific motif containing an alanine or valine at the N-terminus (P1 position), an arginine or proline at the P3 position, and a hydrophobic residue (Phe, Ile, and Tyr) at the P4 position. The BIR2-binding motif is less stringent. Although it still requires an N-terminal alanine, it tolerates a wide variety of amino acids at P2-P4 positions. The BIR1 failed to bind to any peptides in the library. SPR analysis of individually synthesized peptides confirmed the library screening results. Database searches with the BIR2- and BIR3-binding consensus sequences revealed a large number of potential target proteins. The combinatorial library method should be readily applicable to other BIR domains or other types of protein modular domains.