A combinatorial phosphotyrosyl (pY) peptide library was screened to determine the amino acid preferences at the pY+4 to pY+6 positions for the four SH2 domains of protein-tyrosine phosphatases SHP-1 and SHP-2. Individual binding sequences selected from the library were resynthesized and their binding affinities and specificities to various SH2 domains were further evaluated by SPR studies, stimulation of SHP-1 and SHP-2 phosphatase activity, and in vitro pulldown assays. These studies reveal that binding of a pY peptide to the N-SH2 domain of SHP-2 is greatly enhanced by a large hydrophobic residue (Trp, Tyr, Met, or Phe) at the pY+4 and/or pY+5 positions, whereas binding to SHP-1 N-SH2 domain is enhanced by either hydrophobic or positively charged residues (Arg, Lys, or His) at these positions. Similar residues at the pY+4 to pY+6 positions are also preferred by SHP-1 and SHP-2 C-SH2 domains, although their influence on the overall binding affinities is much smaller compared with the N-SH2 domains. A structural model was generated to qualitatively interpret the contribution of the pY+4 and pY+5 residues to the overall binding affinity. Examination of pY motifs from known SHP-1 and SHP-2-binding proteins shows that many of the pY motifs contain a hydrophobic or positively charged residue(s) at the pY+4 and pY+5 positions.