Interactions between Src homology 2 (SH2) domains and phosphotyrosine sites regulate tyrosine kinase signaling networks. Selective perturbation of these interactions is challenging due to the high homology among the 120 human SH2 domains. Using an improved phage-display selection system, we generated a small antibody mimic (or 'monobody'), termed HA4, that bound to the Abelson (Abl) kinase SH2 domain with low nanomolar affinity. SH2 protein microarray analysis and MS of intracellular HA4 interactors showed HA4's specificity, and a crystal structure revealed how this specificity is achieved. HA4 disrupted intramolecular interactions of Abl involving the SH2 domain and potently activated the kinase in vitro. Within cells, HA4 inhibited processive phosphorylation activity of Abl and also inhibited STAT5 activation. This work provides a design guideline for highly specific and potent inhibitors of a protein interaction domain and shows their utility in mechanistic and cellular investigations.